|
Chapter Two: Framework for the Assessment
This chapter further discusses the rationale for recommendations, especially those that reflect a change from current policy. Content Areas Since its first mathematics assessments in the late 1970s and early 1980s, NAEP has regularly gathered data on students’ understanding of mathematical content. Although the names of the content areas in the frameworks, as well as some of the topics in those areas, may have changed somewhat from one assessment to the next, there remained a consistent focus toward collecting information on student performance in five key areas:
The framework for the 2005 mathematics assessment is anchored in these same five broad areas of mathematical content:
These divisions are not intended to separate mathematics into discrete elements. Rather, they are intended to provide a helpful classification scheme that describes the full spectrum of mathematical content assessed by NAEP. Classifying items into one primary content area is not always clear cut, but doing so brings us closer to the goal of ensuring that important mathematical concepts and skills are assessed in a balanced way. At grade 12, the five content areas are collapsed into four, with geometry and measurement combined into one. This reflects the fact that the majority of measurement topics suitable for 12th-grade students are geometrical in nature. Separating these two areas of mathematics at grade 12 becomes forced and unnecessary. It is important to note that certain aspects of mathematics occur in all of the content areas. The best example of this is computation. Computation is the skill of performing operations on numbers. It should not be confused with the content area of NAEP called Number Properties and Operations, which encompasses a wide range of concepts about our numeration system (see chapter 3 for a thorough discussion). Certainly the area of Number Properties and Operations includes a variety of computational skills, ranging from operations with whole numbers to work with decimals and fractions and finally real numbers. But computation is also critical in Measurement and Geometry, such as in calculating the perimeter of a rectangle, estimating the height of a building, or finding the hypotenuse of a right triangle. Data analysis often involves computation, such as calculating a mean or the range of a set of data. Probability often entails work with rational numbers. Solving algebraic equations usually involves numerical computation as well. Computation, therefore, is a foundational skill in every content area. While the main NAEP assessment is not designed to report a separate score for computation, results from the long-term NAEP assessment can provide insight into students’ computational abilities. Mathematical Complexity of Items The framework used for the 1996 and 2000 NAEP included three dimensions: mathematical content, mathematical abilities, and power (reasoning, connections, and communication). That framework was intended to address the primary need of making sure that NAEP assessed an appropriate balance of mathematical content and at the same time assessed a variety of ways of knowing and doing mathematics. The abilities and power dimensions were not intended for reporting, but rather to provide for a wide range of mathematical activity in the items. There were many laudable features of that framework. Notions of conceptual understanding, procedural knowledge, and problemsolving sent a strong message about the depth and breadth of engaging in mathematical activity. The dimensions of mathematical power gave further emphasis to the idea that certain activities cut across content areas. At the same time, there was an acknowledgement that the dimension of mathematical abilities proved somewhat difficult for experts to agree on, relying as it does on inferences about students’ approaches to each particular item. The intent of the 2005 framework is to build on the former framework, retaining its strengths while addressing some of its weaknesses. The purpose remains the same: to make sure that NAEP assesses an appropriate balance of content as well as a variety of ways of knowing and doing mathematics. The major change is to create a second dimension of the framework based on the properties of an item, rather than on the abilities of a student. Mathematical complexity of an item answers the question, “What does the item ask of the students?” Each level of complexity includes aspects of knowing and doing mathematics, such as reasoning, performing procedures, understanding concepts, or solving problems. The levels are ordered, so that items at a low level would demand that students perform simple procedures, understand elementary concepts, or solve simple problems. Items at the high end would ask students to reason or communicate about sophisticated concepts, perform complex procedures, or solve nonroutine problems. Ordering of the levels is not intended to imply a developmental sequence or the sequencing in which teaching or learning occur. Rather, it is a description of the different demands made on students by particular test items. See chapter 5 for further discussion of the levels of mathematical complexity. Distribution of Items The distribution of items among the various mathematical content areas is a critical feature of the assessment design, as it reflects the relative importance and value given to each of the curricular content areas within mathematics. As has been the case with past NAEP assessments in mathematics, the categories have received differential emphasis at each grade, and the differentiation continues in the framework for the 2005 assessment. Table 1 provides the recommended balance of items in the assessment by content area for each grade (4, 8, and 12) in the 2005 assessment. Note that the percentages refer to numbers of items, not the amount of student testing time (see chapter 5 for recommendations on item formats and student testing time). Table 1. Percentage Distribution of Items by Grade and Content Area
The percentages in grade 4 are not different from those recommended in the framework for the 1996 and 2000 assessments. Change was not recommended, as these numbers continue to be a reasonable reflection of the relative weights of each of these content areas at that grade level. At grade 8, new number concepts occur in the form of more advanced work with properties and operations on rational numbers (fractions and decimals) and more sophisticated work in number theory. However, much of the work in numbers happens in the context of other content areas, such as Measurement and Data Analysis and Probability. Grade 8 also has an increased emphasis on informal algebraic concepts and on Geometry. Therefore, the percentages show a slight increase in Algebra and a corresponding slight decrease in Number Properties and Operations at grade 8 compared with the percentages of the previous framework. More students are taking higher levels of mathematics in high school. According to data from the 2000 NAEP, approximately 79 percent of 12th-grade students have taken the equivalent of 2 years of algebra and 1 year of geometry. Because NAEP needs to assess the full range of content in mathematics, the percentages for 12th grade have been adjusted, with primary emphasis on Geometry/Measurement and Algebra. At this grade level, the majority of measurement topics are geometrical in nature and the distinction between Geometry and Measurement becomes blurred, so the 2005 framework calls for these two content areas to be combined into one. The percentages also show the increased importance of Data Analysis and Probability in the secondary school curriculum. The majority of work in Number Properties and Operations is done in the context of the other content areas, so it receives a decreased emphasis. Special Study at Grade 8 The past decade has brought major changes to the middle school mathematics curriculum in the form of more students taking courses called algebra and more algebraic concepts being taught in other mathematics courses at grades 7 and 8. From the 2000 NAEP assessment, we know that 31 percent of eighth graders reported taking prealgebra and 26 percent reported taking either first- or second-year algebra. We have also learned from research and from international studies such as Trends in International Mathematics and Science Study that the topic of proportionality is particularly challenging for students around the world. Proportionality (such as “a is to b as c is to d”) is foundational for understanding many algebraic concepts, such as rate of change, variation, linear relationships, slope, and functions, as well as concepts in other advanced mathematics courses and in other disciplines. Taken together, these two findings form the basis for the purpose of a special study of grade 8 students: to examine the depth and breadth of eighth-grade students’ understanding of proportionality and other fundamental topics in algebra. The special study would be designed to answer two research questions:
Items for the special study should be administered to a random sample of eighth-grade students who also participate in regular NAEP. The items would be administered along with the main NAEP items, but in separate blocks, so that only a subset of students is taking special study items. Because the main NAEP assessment is designed to assess all five content areas of mathematics at grade 8, it is not possible to obtain indepth data about students’ understanding of fundamental topics in algebra or proportionality. This special study would provide an opportunity to more closely study students’ mathematical development in proportionality and related topics in algebra during the critical middle school years. It would also yield a clearer description of the depth and breadth of grade 8 students’ knowledge in these areas, which would be helpful in the design of future NAEP frameworks. Calculators At each grade level, approximately two-thirds of the assessment measures students’ mathematical knowledge and skills without access to a calculator; the other third of the assessment allows the use of a calculator. The assessment contains blocks for which calculators are not allowed, and calculator blocks, which contain some items that would be difficult to solve without a calculator. The type of calculator students may use on a calculator block varies by grade level, as follows:
No items on the 2005 NAEP at either grade 8 or grade 12 will be designed to provide an advantage to students with a graphing calculator. The estimated time required for any item should be based on the assumption that students are not using a graphing calculator. In determining whether an item belongs on a calculator block or a noncalculator block, the developer should take into consideration the technology available to students and the measurement intent of the item. The content and skills being assessed should guide whether an item should be considered calculator active and whether it should go in a calculator block. For example, a multiple-choice item asking students to select the graph that represents a given equation should be on a noncalculator block, if the intent of the item is to measure students’ ability to recognize the graph of a given equation and if the equation is in a form that is readily entered into a graphing calculator to obtain a graph (for example, y = –x2).
|