American colonists first began establishing elementary schools for their children in the early 17th century. These schools were private, and only the wealthiest families could afford to enroll students in them. The main purpose of these early schools was to teach the students religion due to its major role in colonial life. Reading, writing, and arithmetic weren't considered as important. Many children were taught at home by governesses or tutors. Wealthy children eventually were sent to Europe to receive a "higher education" there.

Before 1642, only 10 percent of the young children in the colonies attended school. That year, the colony that would eventually become Massachusetts passed a law stating that parents must teach their children to read. By 1647, the colony mandated that every town with 50 families or more must establish an elementary school. This paved the way for widespread education for children. However, it wasn't until more than 200 years later, in 1852, that the state of Massachusetts passed the first law in the country making school attendance compulsory. By 1918, all states had such laws.

The earliest secondary school was located in the Massachusetts Bay Colony. The Boston Latin School opened in 1635 with the purpose of training children, primarily boys, to become well-read members of the clergy. Like the Boston Latin School, many of the early secondary schools focused on the language of Latin. In the 1700s, the colonists, led by Benjamin Franklin, founded another type of secondary school called the academy. Academies offered a wide variety of subjects in addition to religion, but they were all private schools that charged tuition and catered to the sons and daughters of affluent families.

Public education did not really take hold until the early 1800s, after the Revolutionary War had ended and the country was beginning to unite. Boston was the site of the first public high school, which opened in 1821. After fighting the war, many Americans wanted children to learn about patriotism and about becoming morally upstanding individuals. Many also wanted to de-emphasize religion in the schools. The growing spirit of unity in the country had another important effect on public education, namely the movement toward using standard texts in all schools.

The first college in the United States was Harvard, founded in 1636. By 1833, Ohio boasted the country's first coeducational college, Oberlin.

The concept of kindergarten (meaning children's garden in German) came to the United States from Germany in 1856, when Margaretha Schurz opened a private kindergarten in Watertown, Wisconsin. The first public kindergarten was established in St. Louis in 1873. In 1901, the first public two-year college in the United States, Joliet Junior College, was founded in Illinois. Today, there are 1,462 community colleges in the United States. Although World War II caused many people to leave school, the public school system continued to expand and take on more significance, even in the political and social arenas. One of the most important educational reforms of the 20th century was the 1954 Supreme Court decision forbidding racial segregation in public schools.

School systems have suffered many crises since the beginning of the 21st century. Reports by the National Commission on Excellence in Education stress the widespread public perception that something is seriously amiss in the existing educational system. Many feel that educational systems fall far short of the goal of cultivating a learning society and that U.S. students lag far behind their counterparts in other developed nations.

The No Child Left Behind (NCLB) Act was enacted into law in 2002. The Act did the following: (1) sets achievement goals for teachers and paraprofessionals; (2) holds states to higher accountability for the educational achievement of their students in exchange for more flexibility in how they can use federal education funds; (3) stresses the importance of using scientific research to determine what educational programs and practices are most effective for students and educators; and (4) gives parents with children in schools that do not meet state standards for at least two straight years the option to transfer their children to a better-performing public school or public charter school. The No Child Left Behind Act is an extension of the Elementary and Secondary School Act of 1965. It was to be considered for reauthorization in 2010, but political differences delayed consideration and reauthorization. In 2012, the Obama administration offered flexibility to states regarding the requirements of the NCLB in exchange for the states' rigorous plans to meet achievement goals. In 2015, the Senate passed the Every Student Succeeds Act (ESSA), in replacement of the NCLB Act. The ESSA is now the main education law for all U.S. K–12 public schools, ensuring that public schools provide quality education for all students.

There has been a renewed focus on the quality of U.S. career and technical preparation schools, which prepare students at the secondary level to enter the workforce, usually in a skilled trade. There is a renewed focus on the role of academics in these programs: students in secondary technical and career programs now take the same standardized tests as students in traditional high schools. These programs now include a stronger emphasis on workplace skills, such as communication and problem solving.

In elementary and secondary public education in the 21st century, current issues are those focusing on improving the quality of the education delivered to students, including looking at class size, access to early childhood education, and others. In recent years, elementary and secondary education has undergone a transformation to adhere to the requirements of the Common Core State Standards Initiative. This effort to define what students should learn and how they should learn it proved controversial.