2010 NAEP Geography Report

National Assessment of Educational Progress

Geography 2010 Report Card

Jack Buckley 

Commissioner
National Center for Education Statistics

Today I am releasing the results of the 2010 Geography assessment. This is our first Geography assessment since 2001. The assessment measures students' geography knowledge and skills, and is organized around content areas that describe specific geography subject matter and cognitive areas that reflect different levels of understanding geography.

The assessment was administered in early 2010. We have national results for grades 4, 8, and 12. Approximately 7,000 fourth-graders took the assessment, while both the grade 8 and grade 12 samples were larger—9,500 or more.

Overall results are based on the performance of both public and private school students. At grades 4 and 12, participation rate standards for separate reporting of private school students were not met, so we only have private school results at grade 8 for 2010.

Student performance is presented in two ways—average scale scores, with a single 0-500 scale for all three grades, and separate achievement levels for each grade. The NAEP achievement levels—Basic, Proficient, and Advanced—are set by the National Assessment Governing Board, which sets policy for NAEP. NAEP scale scores tell us what students know and can do, while the NAEP achievement levels provide standards for what students should know and be able to do.

For both scale scores and achievement level performance, we will be making comparisons back to previous assessments in 1994 and 2001. When making these comparisons, we must remember that all NAEP results are based on samples. This means that there is a margin of error associated with every score and percentage. When discussing changes in student performance—either increases or decreases—we only discuss those that are statistically significant—those that are larger than the margin of error. In the tables and figures that follow, an asterisk is used to indicate statistically significant differences when comparing scores from previous assessments to 2010.

The Geography assessment is guided by a framework that combines key physical science and social science aspects of geography, and focuses on what geography students should know to be competent and productive 21st century citizens. The National Assessment Governing Board oversees the development of the assessment framework. The assessment groups questions into the three content areas: Space and Place, Environment and Society, and Spatial Dynamics and Connections. The relative percentage of each of the content areas in the framework is the same for grades 4, 8, and 12: 40 percent for Space and Place, 30 percent for Environment and Society, and 30 percent for Spatial Dynamics and Connections.

The three types of cognitive skills for geography are identified in the framework as Knowing, Understanding, and Applying. The relative percentages of each of the cognitive skills are different for grades 4, 8, and 12. The emphasis on Knowing decreases in the upper grades, while the emphasis on Applying increases.

The geography cognitive skills are defined as follows:

• Knowing
  
  Questions in this area ask students: What is it? Where is it? Students should be able to observe different elements of the landscape and answer questions by recalling, for example, the name of a place.
  
  
• Understanding
  
  The questions about Understanding ask students: Why is it there? How did it get there? What is its significance? Students should be able to attribute meaning to what has been observed and explain events.
  
  
• Applying
  
  Finally, students are asked to apply what they've learned. How can knowledge and understanding be used to solve geographic problems? Students should be able to classify, hypothesize, use inductive and deductive reasoning, and form problem-solving models.

Grade 4 Results

The average score for grade 4 students in 2010—213—was higher than on either prior assessment. When we look at student performance at various percentiles, we get a more detailed picture of student performance. Scores improved for low-performing students, at the 10^th and 25^th percentiles, and for those in the middle, compared to both 1994 and 2001. For higher performing students (those at the 75^th and 90^th percentiles), however, scores did not change significantly in comparison to either previous assessment.

The increases in performance for lower-performing students are reflected in the grade 4 achievement level results. Fifty-eight percent of fourth-graders were in the Basic range in 2010, compared to 48 percent in 1994 and 52 percent in 2001. At the same time, the percentage at Advanced fell from 3 percent in 1994 to 2 percent in 2010.

Scores for White, Black, Hispanic, and Asian or Pacific Islander students were higher in 2010 than in 1994. Since 1994, the 26-point increase for Black students and the 19-point increase for Hispanic students were larger than the 6-point increase for White students. When we compare scores in 2010 to scores in 2001, we see increases for White, Black, and Hispanic students.

Comparisons could not be made to prior years for American Indian/Alaska Native students because the NAEP samples for 1994 and 2001 for these students were not large enough to permit the reporting of reliable results.

In 2010, average scores for White and Asian or Pacific Islander students were higher than the scores for other racial/ethnic groups.

Achievement level results are available for the four racial/ethnic groups for whom we have trend data. The percentages of Black and Hispanic students at Basic were higher than in either previous assessment. For Black students only, the percentage at Proficient was higher as well, comparing 2010 to 1994.

For White students, the percentage at Basic was higher in 2010 than in 1994. For Asian/Pacific Islander students, the percentage at Basic did not change significantly, but the percentage below Basic declined, falling from 28 percent in 1994 to 13 percent in 2010. For White, Black, and Hispanic students, the percentage below Basic was lower than in either previous assessment year.

Racial/ethnic gaps narrowed in 2010. Scores for both Black and White students increased when compared to 1994. Because the score increases for Black students were larger than the increases for White students, the 31-point gap in 2010 was narrower than the previous gaps in 1994 and 2001.

With respect to the White-Hispanic score gap, we see a similar pattern: larger score increases for Hispanic students resulted in a 27-point gap for 2010 that was narrower than in previous assessments.

Scores were higher in 2010 for both male and female students than in either prior assessment. The 4-point difference in scores in 2010 was statistically significant and was not measurably different from the 5-point gaps in 2001 and 1994.

When NAEP assesses students, we also ask their teachers about their instructional practices. Among other things, we asked teachers how often they instructed their students in a variety of topics related to geography. Most students had teachers who said they instructed their students at least once or twice a month in these topics: other countries and cultures, environmental issues, maps and globes, natural resources, space and place, or spatial dynamics.

Grade 8 Results

At grade 8 there was no change in the average score in 2010 from either 1994 or 2001. When examining scores at selected percentiles, we see that the average score for students at the 10th percentile was 7 points higher than in 1994, and 4 points higher at the 25^th percentile. Comparing scores in 2010 to those in 2001, we again see a 7-point increase at the 10^th percentile.

Even though the average score for eighth-graders did not change from 1994 to 2010, the percentage of students at the Basic achievement level did change, increasing to 47 percent, as compared to 43 percent in both 1994 and 2001. The percentage below Basic fell from 29 percent in 1994 to 26 percent in 2010. However, the percentage at Advanced also fell, from 4 percent in 1994 to 3 percent in 2010.

The White-Black score gap in geography—31 points in 2010—was narrower than in either 1994 or 2001. The average scores for both Black and White students were higher in 2010 than in 1994, but the 12-point increase for Black students was larger than the 3-point increase for White students.

The White-Hispanic score gaps for the three previous administrations of the geography assessments did not change significantly. Average scores for Hispanics in 2010 were higher than in either previous assessment, but the increases were not large enough to cause a significant change in the size of the gap.

We asked eighth-grade students about the frequency with which they studied certain topics in geography. Comparing 1994 to 2010, increased percentages of students reported that they frequently studied countries and cultures and environmental issues (examples given in the questionnaire were pollution and recycling). The decline occurred for natural resources (exemplified in the questionnaire by oil, forests, and water). In 2010, 30 percent of students reported that they studied natural resources at least once a week, unchanged from 1994 but lower than the 33 percent shown for 2001.

Grade 12 Results

In 2010 the average score for grade 12 students—282 points—was lower than in 1994 and not significantly different from 2001. When considering score changes since 1994 by percentile, there was an increase of 3 points at the 10th percentile, and decreases at the 50^th, 75^th, and 90^th percentiles. Since 2001, there were also decreases for middle and higher-performing students.

In terms of achievement levels, there was an increase in 2010 in the percentage of students at Basic, compared to both prior assessment years, and a decrease in the percentage at Proficient. Since 2001, there was a decline in the percentage at Advanced. The increase for the percentage at Basic could be due to both score increases for lower-performing students and score decreases for higher-performing students.

There was no significant change in the White-Black score gap at grade 12 in 2010. In addition, there were no significant changes in scores for either White or Black students. There were also no significant changes in scores for White or Hispanic students, nor in the Black-White or Hispanic-White gap.

Male students scored higher than female students in all three administrations. The 5-point gap in 2010 was not significantly different from the gaps in the prior two assessments. However, the average score for male students in 2010 was lower than in 1994, falling by 3 points.

We asked grade 12 students the same questions as we asked students in grade 8 about the geography topics they studied in class and the frequency with which they studied those topics. For grade 12 students, the percentages who said they studied natural resources, countries and cultures, and environmental issues at least once a month were higher in 2010 than in either prior assessment.

Summary

At grade 4, the overall score was higher than on either prior assessment. Scores also improved for low- and middle-performing students, i.e., those at the 10^th, 25^th, and 50^th percentiles.

At grade 8, the overall score in 2010 was not significantly different from either prior score. Scores for low-performing students were higher in 2010 than they had been in 1994.

At grade 12, the overall score was lower than in either prior assessment. This was true for middle- and higher-performing students as well. Improvement since 1994 was seen only for students at the 10^th percentile.

Scores were higher in 2010 than in either previous assessment year for Black and Hispanic students at grades 4 and 8. This was also true for White students at grade 4 only.

For White students at grade 8, and for Asian or Pacific Islander students at grade 4, scores in 2010 were higher than in 1994.

Earlier this year, I released results for two other subjects assessed in 2010 that come under the broad heading of social studies, civics and U.S. history. As geography is the third and last of these subjects assessed in 2010, it is useful to summarize results across the three social studies subjects.

At grade 4, scores in 2010 were higher than in any earlier assessment for both civics and geography. For U.S. history, the grade 4 average score was higher in 2010 than in 1994.

At grade 8, scores in 2010 were not significantly different from any earlier assessment for both civics and geography. For U.S. history, the score in 2010 was higher than in both previous assessments.

At grade 12, the average score in 2010 was lower than in 2006 for civics, higher than 1994 for U.S. history, and lower than in 1994 for geography.

The 2010 Geography Report Card provides all of the information about the geography assessment I described today and much more. In addition, the initial release website gives extensive information on the performance of students, access to released assessment questions through NAEP's Questions Center, and access to all the variables collected in NAEP through the NAEP Data Explorer, our online data-analysis tool.

In conclusion, I would like to offer my sincere thanks to all the students, teachers, and schools who participated in the 2010 Geography assessment.

National Assessment of Educational Progress

Geography 2010 Report Card

Roger M. Downs 

Professor of Geography, The Pennsylvania State University

As a geography educator and a veteran of all three NAEP geography assessments, it is an honor to have this opportunity to help focus attention on an increasingly important school subject. I want to highlight some key results from the 2010 NAEP geography report card and put them into context by asking why they might have occurred and what they might mean for K-12 geography education in the future.

The overall pattern of student performance in this report shows improvements at grade 4, no change at grade 8, and a slight decline at grade 12. This pattern reflects the status of geography in America's schools. Geography is taught as part of social studies in grades K-4 and that curricular arrangement often extends into grades 5 and 6. Geography typically appears in grades 7 or 8 as a stand-alone course with a "world cultures" label. In high school, geography is rarely taught as a stand-alone course and, at best, it is infused with another subject. The only exception is the rapid growth of Advanced Placement Human Geography with more than 68,000 AP test takers in 2010, a 35 percent increase compared to 2009.

Despite the relative lack of attention to geography in America's schools, the 2010 NAEP Geography Report Card contains some encouraging news. First, the significant increase in performance by students in grade 4 is important because these students represent the next generation moving through U.S. schools. They will progress into higher grades with a stronger grounding in geography. Second, the increase in scores for the lowest-performing students at all three grades means that the gap between the lowest and highest performing students is narrowing. Third, another gap is narrowing. It is especially pleasing to see the significant narrowing of the gap in average performance between Black and White students at grades 4 and 8. These two narrowing gaps remind us that geography is accessible and important to all students, and both groups should have the foundations they need to be informed and productive citizens.

The report, however, also contains some discouraging news. First, average performance at both grades 8 and 12 did not improve over average performance reported in either 1994 or 2001. In fact, at grade 12, there was a decrease from 1994. Second, the gap persists between the performance of boys and girls. For all three NAEP Geography reports, boys have out-performed girls in terms of average scores by grade and in terms of percentage at or above the Proficient level by grade. While this gap is real, it is important not to overstate the difference. Although the difference in average scores between boys and girls is statistically significant, it is small: the two distributions overlap to a large measure. Boys and girls are equally capable of mastering and enjoying geography.

In summary, therefore, we see a picture of positive change, no change, and even slight decline across the grades. What might be responsible for this mixture of results?

First, given the current commitments of curriculum time, the NAEP Geography results meet the expectations–if not the hopes–of geography educators. To the extent, however, that classroom time becomes an even more precious and scarce commodity, geography, with subjects such as history and the arts, is losing out in the zero-sum game that results from high-stakes testing. In 1994, an influential report about time allocation in the curriculum had the provocative title of Prisoners of Time. Sadly, geography is in danger of becoming a casualty of time.

Second, many of the drivers of change are not unique to the United States. In many countries we see a similar shift in educational priorities leading to the reallocation of time resources, and a comparable emphasis on high-stakes testing. We can also see similar patterns of results, as in the gap in performance between boys and girls.

Given the NAEP Geography 2010 results and current educational priorities in the United States, what are the implications for geography education that will take place before the next NAEP Geography in 2014?

Time in the classroom is allocated on the basis of a subject's perceived importance to the education of the next generation of students. In the case of geography, the rationale for importance has centered on the idea of a literate high school graduate who is able to meet the demands of citizenship. While geography begins with place location knowledge, the focus on understanding relationships between people and their environment is increasingly crucial in a world in which the local is connected to the global, and global events affect local places. Geography is all about understanding the connections between people, places and environments. As the economic and cultural forces of globalization and the impacts of global environmental change are felt by everybody everywhere, the case for geography seems both obvious and inescapable.

And yet geography's role in the curriculum is limited and, at best, static. That is ironic given the convincing case that can be made for the importance of geographic literacy. But it is doubly ironic given a world in which adults and now children have smart phones and tablets that can download maps on the fly, provide directions to places, and give your location to your friends. We are entering a world in which GPS, the global positioning system, is changing our ability to know about the locations of ourselves and others. Online mapping sites such as Google Earth allow students to explore any location in the world as a map or a satellite image and to change the scale of the map or image. Behind many web sites that provide locational information are Geographic Information Systems (GIS) that manipulate geospatial data. Through volunteered geospatial information, students using GIS can help to create maps that are more up-to-date and accurate than those of official mapping agencies. And geospatial data are becoming commonplace in activities ranging from getting travel directions to finding the nearest store.

I hope to see a NAEP report in which there are illustrations of students working with computers, geospatial tools, and geographic thinking skills to solve problems not just in geography but in biology, history, ecology, economics and beyond. Thus, the grade 12 question on population density in Australia and Libya on page 47 of the report, for example, could be reframed using a GIS program. Students could be presented with a map of population density in either country and asked to overlay the map with one of a series of other maps—topography (elevation and drainage), climate, resources—in order to explain the pattern of population density.

The geospatial revolution has radically changed our capacity to analyze, represent and understand our world. It offers powerful tools that have applications not just in geography as a school subject but across the entire curriculum and, equally importantly, in our daily lives beyond school. A high school graduate must be able to think spatially and understand how to use geospatial tools in ways that are appropriate and responsible. Geography is, therefore, increasingly important in understanding our world and in coming to terms with it. Its value goes beyond just fostering informed citizenship: it offers career and lifelong learning skills and it allows us that ultimate existential understanding: Who we are is where we are.

For those reasons, I hope the next NAEP report will demonstrate progress in geographic understanding at all grades, for all students. Thank you.

National Assessment of Educational Progress

Geography 2010 Report Card

Shannon Garrison 

Member, National Assessment Governing Board;
Fourth-Grade Teacher, Los Angeles, CA

As Dave Gordon mentioned in the introductions, I am a fourth-grade teacher at Solano Avenue Elementary School in downtown Los Angeles. We are just a few blocks from Dodger Stadium and close to Chinatown.

Most of our students are Asian or Latino and they come from low-income homes. Many are English Language Learners. There are approximately 10 different languages spoken by the students at Solano, and their family histories cover a great deal of geography. Yet, many of these children have never travelled anywhere outside their own neighborhood.

This past spring, I took students on a field trip to the Manzanar National Historic Site, which was an internment camp for Japanese Americans and people of Japanese descent during World War II. It's 225 miles north of Los Angeles in the eastern Sierra Nevadas, about a four-hour bus ride from LA. Many of the students kept asking if we were still in California even though we had studied where the camp was on a map. They couldn't comprehend that we were still in California, that our state is so big and so varied.

Geography is the study of our planet earth—how its lands, seas, and people interact, and of the impact that its different peoples, economies, and societies have upon one another.

Geography—along with history, civics and economics—is part of the social studies. Each of these subjects is tested separately by NAEP in order to give a clear report on the subject-matter knowledge and skills that students have attained. Yet, obviously, these subjects are intertwined. Geography provides the context for all the others.

Students need to understand the relationships between the environment and the societies that develop within it. They need to study the relationships between various peoples and cultures, and make the connections between the location of natural resources and economic development.

All of these relationships provide context for understanding the social, political, and economic structures that exist in our world today.

The NAEP Geography assessment goes well beyond place-name geography, though students are expected to know the names and locations of many places. To reach the Proficient achievement level on the assessment, students must be able to apply what they know, to use important concepts, and to analyze and explain specific situations. It is in these application and analytical skills that too many of our students fall short.

As Commissioner Buckley has told you, the average score on the NAEP Geography assessment has increased over the past decade and a half at grade 4, stayed flat at grade 8, and declined at grade 12. All of the gains at grade 4 have taken place at the Basic achievement level or below—at the lower end of the achievement distribution. Since about the year 2000, there have been similar gains at the fourth-grade Basic level in NAEP Civics and U.S. History.

These improvements may all reflect another trend in NAEP, the gain in basic reading skills, as reported by the fourth-grade reading assessment over the past decade. The improvement in basic reading skills would certainly impact the lower-performing students since if they can read the questions easily they have a much better chance of answering them correctly. However, to reach Proficient requires more detailed knowledge of geography and critical thinking skills. Unfortunately, there have been no gains at the Proficient level.

The proportion of students with at least a Basic-level knowledge of geography is reasonably high—about 70 to 80 percent at grades 4, 8 and 12. But only about a quarter of the students reach the Proficient level, and that small proportion has not changed significantly or declined.

The situation at 12^th grade is the most disappointing. There, the proportion of students reaching Proficient has dropped steeply—from 27 percent in 1994 to just 20 percent last year. To attain that standard, which was set by the Governing Board, students should be able to interpret maps, to discuss the physical and cultural features of major world regions, and to discuss the economic, political, and social factors that define various parts of the earth. Yes, the Proficient standard is challenging, but it is one that students need to reach in order to understand our complex world.

More 12^th graders report that they study geography topics at least once a month, but very few seem to develop the analytical and conceptual framework they need—or much in-depth knowledge of the subject.

In elementary school, many teachers integrate geography into the other subjects they teach. It is not only part of social studies; it also can be referred to and developed in reading and science—and even in math. Children are always interested in other places and other people—no matter what their socio-economic background or how limited their own travel and experience. Incorporating examples and concepts from geography can make any subject come alive.

But in middle and high school, geography is often the unclaimed subject. In many districts and schools, the responsibility for teaching geography is unclear. The data seem to indicate that students are not receiving sufficient, quality instruction in geography, and as an educator and citizen, this concerns me.

Besides the sample questions in the Geography Report Card, several dozen more test items have been released today. They are available to the public online through the NAEP Questions Tool. I would like to talk about a few of the questions, which offer some insight into what students know and how clearly they can think and reason.

At grade 4 there is a question that asks students to determine distance on a map. That's a basic skill. But because of the dependence on technology, the ability to read a map seems to be becoming a lost art. It shouldn't be lost because everyone needs to have some idea of the spatial relationships and distances between different places, and that can be best understood by reading a map. Sadly, on this question, only about a third of the students chose the correct answer, which is D—140 miles.

Here's another question where the percentage of correct responses shocked me. I couldn't believe that only 51 percent of fourth-graders could put these places in descending order of size—North America, the U.S.A., California and Los Angeles. As a fourth-grade teacher, this is very troubling.

At 8^th grade there are several questions which show the strengths and weaknesses of what students know and can do in geography. The first is a line graph, showing the changes in urban and rural population in the United States during the 19^th and 20^th centuries. Most students could read the graph. When asked a factual question about it, 83 percent answered correctly. But when asked to give two reasons why the population changes occurred, performance fell off: 26 percent gave one acceptable reason; just 4 percent gave two reasons; and the remaining 70 percent of test papers were wrong, blank or off-task.

About half of the eighth-grade students answered correctly a multiple-choice question about why sod houses were built on the American Great Plains. But only 9 percent could write a full explanation of the population distribution in Egypt, which they were shown on a map.

At 12^th grade about 80 percent correctly answered a multiple-choice question on the main reason why people move from one country to another—economics. But only 12 percent could give even a partial explanation, based on a diagram, of how changes in transportation have influenced urban growth.

It seems pretty clear from these examples that most students have some basic skills and some basic facts, but they fall well short of being able to marshal enough information and general concepts to give responsive answers to some straight-forward questions.

The basic skills are important. The improvements shown in basic reading and at the Basic level in fourth-grade geography are significant. But our schools must go beyond that to develop the insights and analytical abilities that our students need to understand the world.

As a teacher, I realize the power of geography. It provides the context for understanding many of the complex social, political and economic relationships that exist in our world. Geography is easily integrated into any subject, and it can provide the engaging, real world examples that make learning meaningful. I believe the social studies, including geography, are crucial for our students and for our schools. The NAEP test itself shows some of the richness of the field. Unfortunately, it also shows that too many students still fall far short of the knowledge and understanding they need.

National Assessment of Educational Progress

Geography 2010 Report Card

David W. Gordon

Superintendent of the Sacramento (Cal.) County Office of Education

David W. Gordon is Superintendent of the Sacramento (Cal.) County Office of Education, which directly serves more than 30,000 students and provides financial oversight and support services to more than 235,000 students in 16 school districts. Mr. Gordon has been a member of the National Assessment Governing Board since 2003 and chairs the Board's Reporting and Dissemination Committee.

From 1995 to 2004, Mr. Gordon served as Superintendent of the fast-growing Elk Grove Unified School District, where he was responsible for 55 schools and budgets totaling $500 million. He had previously worked for 17 years at the California State Department of Education, rising to the position of Deputy State Superintendent of Public Instruction. Mr. Gordon began his career in 1968 as an elementary school teacher in New York's South Bronx.

Mr. Gordon currently serves as Co-Chair of the California Campaign for the Civic Mission of Schools. He has also served as an Associate in Education at Harvard University, a visiting scholar at Stanford University, and a visiting professor at the University of California, Riverside. He has presented at the White House and testified before the U.S. Congress and state legislatures, and he speaks frequently at major conferences.

Mr. Gordon holds an Ed.M. and a Certificate of Advanced Study in Educational Administration from Harvard University and a B.A. from Brandeis University.