Oct. 15 Deadline for Achievement Levels Comments
SUBMISSIONS ARE NOW CLOSED
REQUEST FOR PUBLIC COMMENT: REVISED POLICY ON DEVELOPING STUDENT ACHIEVEMENT LEVELS FOR NAEP
The National Assessment Governing Board (Governing Board) is soliciting public comment for guidance in finalizing a revised policy on Developing Student Achievement Levels for the National Assessment of Educational Progress (NAEP).
Under P.L. 107-279, the Governing Board is authorized to formulate policy guidelines for NAEP. The legislation specifies that the Governing Board is to develop appropriate student achievement levels for each subject and grade tested, as provided in section 303(e). Such levels are determined by identifying the knowledge and skills that can be measured and verified using widely accepted professional assessment standards.
To carry out its legislative mandate, the Governing Board has a policy on setting achievement levels for NAEP. The existing policy was last updated in 1995 and is in need of revision to reflect current best practices in standard setting and to partially address some of the recommendations from a recent evaluation of the NAEP achievement levels. Over the past 1.5 years, the Governing Board and its contractor the Human Resources Research Organization (HumRRO) have been conducting research to guide the policy revision, including the convening of an expert panel.
Voluntary participation by all interested parties is urged. All responses will be taken into consideration before finalizing the revised policy for Board adoption. Comments can be provided via email (NAEPALSpolicy@ed.gov) by October 15, 2018. Comments may also be provided via mail, to be received no later than October 15, 2018, at the following address:
NAEP Achievement Level Setting Policy
National Assessment Governing Board
800 North Capitol Street, N.W., Suite 825
Washington, DC 20002
It is anticipated that the revised policy on Developing Student Achievement Levels for NAEP will be presented for approval at the National Assessment Governing Board quarterly meeting on November 15-17, 2018.
RESOURCE MATERIALS FOR PUBLIC COMMENT:
Proposed Revised Policy on Developing Student Achievement Levels for the National Assessment of Educational Progress
The proposed revised policy can be downloaded here.
Existing Policy on Developing Student Performance Levels for the National Assessment of Educational Progress
The existing policy (adopted in 1995) can be downloaded here.
Governing Board’s Formal Response to the Evaluation of NAEP Achievement Levels
From 2014-2016, the National Academies of Sciences, Engineering, and Medicine conducted an independent evaluation of the NAEP achievement levels. The Governing Board’s formal response to the recommendations put forth in the evaluation noted that the revision of the Board policy on developing achievement levels for NAEP would specify a process and timeline for conducting regularly recurring reviews of the achievement level descriptions and would be explicit about the conditions that necessitate consideration of a new standard setting. More information about the evaluation and the Governing Board’s response can be found here. The final report from the evaluation can be found here.
Summary of Proposed Revisions
Compared to the existing 1995 policy on Developing Student Performance Levels for NAEP, the proposed revised policy reflects:
- Reorganization of principles, streamlining of language, minimization of redundancies
- Minor (non-substantive) edits to the NAEP policy definitions for clarity
- A change in terminology from Proficient to NAEP Proficient to better differentiate the NAEP achievement levels from other common uses of Basic, Proficient, Advanced
- A new principle on periodic review of achievement level descriptions and cut scores, prompted by the Board’s response to the evaluation of NAEP achievement levels
- A new principle to clarify participation of multiple stakeholders at various points throughout process
- A new principle to summarize the role of the Board
- Reference to an interpretative guide that would accompany the release of NAEP results and explain how the achievement levels should (and should not) be used
- Reference to multiple types of achievement level descriptions (ALDs), including reporting ALDs that would be created using empirical data and written in terms of what students do know and can do rather than what students should know and be able to do
- Clarification on the standard setting participants, in particular the non-educator group
- Additional details about the achievement level setting process, including some practices that have become institutionalized over time (e.g., the use of “impact data”)
- Removal of details on implementation directed to staff and contractors, which will instead be included in a “procedures manual”
FOR FURTHER INFORMATION CONTACT:
Sharyn Rosenberg, Ph.D.
Assistant Director for Psychometrics
National Assessment Governing Board
Sharyn.rosenberg@ed.gov
202-357-6940