Skip Navigation
Back to PolicyBot

How Well Are American Students Learning

January 1, 2012
By The Brookings Institution

This edition of the Brown Center Report on American Education marks the first issue of volume three—and eleventh issue over all. The first installment was published in 2000, just as the Presidential campaigns of George W.

book bag

This edition of the Brown Center Report on American Education marks the first issue of volume three—and eleventh issue over all. The first installment was published in 2000, just as the Presidential campaigns of George W. Bush and Al Gore were winding down. Education was an important issue in that campaign. It has not been thus far in the current campaign for the Republican nomination (as of February 2012). And it is unlikely to be a prominent issue in the fall general election. Despite that, the three studies in this Brown Center Report investigate questions that the victor in the 2012 campaign, and the team assembled to lead the U.S. Department of Education, will face in the years ahead.

The first section is on the Common Core State Standards, a project that President Obama has backed enthusiastically. Forty-six states and the District of Columbia have signed on to the Common Core; detailed standards have been written in English language arts and mathematics; and assessments are being developed to be ready by the 2014–2015 school year. The first section attempts to predict the effect of the Common Core on student achievement.

Despite all the money and effort devoted to developing the Common Core State Standards—not to mention the simmering controversy over their adoption in several states—the study foresees little to no impact on student learning. That conclusion is based on analyzing states’ past experience with standards and examining several years of scores on the National Assessment of Educational Progress (NAEP).

States have had curricular standards for schools within their own borders for many years. Data on the effects of those standards are analyzed to produce three findings. 1) The quality of state standards, as indicated by the well-known ratings from the Fordham Foundation, is not related to state achievement. 2) The rigor of state standards, as measured by how high states place the cut point for students to be deemed proficient, is also unrelated to achievement. Raising or lowering the cut point is related to achievement in fourth grade, but the effect is small, and the direction of causality (whether a change in cut point produces a change in test score or vice versa) is difficult to determine. 3) The ability of standards to reduce variation in achievement, in other words to reduce differences in achievement, is also weak.

Common standards will only affect variation between and among states (analysts use the grammatically suspect “between-state” as shorthand for this kind of variation). Achievement variation existing within states is already influenced, to the extent that standards can exert influence, by the states standards under which schools currently operate. Within state variation is four to five times larger than the variation between states. Put another way, anyone who follows NAEP scores knows that the difference between Massachusetts and Mississippi is quite large. What is often overlooked is that every state has a mini-Massachusetts and Mississippi contrast within its own borders. Common state standards only target the differences between states, not within them, sharply limiting common state standards’ potential impact on achievement differences.

The second section of the Report investigates achievement gaps on NAEP. The NAEP has two different tests: the Long-Term Trend NAEP, which began in 1969, and the Main NAEP, which began in 1990. The two tests differ in several respects, but they both carry the NAEP label and both are integral components of “The Nation’s Report Card.”

Achievement gaps are the test score differences between groups of students with different socioeconomic (SES) characteristics: for example, racial or ethnic background, family income, or language status. The second section poses the question: Do the two NAEP tests report similar achievement gaps? Researchers and policy makers are well aware that significant test score gaps exist between SES groups. Researchers try to study them, policy makers try to close them. What NAEP has to say about the magnitude of such gaps plays an important role in the policy arena. The analysis presented in section two indicates that the two NAEPs do in fact differ. The Main NAEP consistently reports larger SES gaps. This is only a preliminary study, a first cut at the data that reveals a general pattern, so the findings must be viewed cautiously. And explanations for the phenomenon are necessarily speculative. More work needs to be done on this topic.

The third section of the report is on international assessments. Interpretations of international test scores are characterized by three common mistakes. The first occurs when a nation’s scores go up or down dramatically and analysts explain the test score change by pointing to a particular policy. The case of Poland’s gains in reading is offered as an excellent example of dubious causality attributed to a single policy. The second mistake stems from relying on rankings to gauge a country’s academic standing. National rankings have statistical properties that can mislead observers into thinking that large differences are small or small differences are large. They can also make growth appear larger or smaller than it really is. Several examples are provided of misinterpretations of rankings and suggestions on how to avoid them. The third mistake is pointing to a small group of high performing nations, often called “A+ countries,” and recommending, with no additional analysis, that their policies should be adopted. The same policies may be embraced by the lowest performing nations or nations in the middle of the distribution. On any test, the entire distribution must be considered, not just scores at the top.

Article Tags