NAEP 2009 Preparedness Standard Setting: Process Report

National Assessment of Educational Progress Grade 12 Preparedness Research Project Judgmental Standard Setting (JSS) Studies: Process Report

WESTED, 2011, GRADE 12

Report on judgmental standard-setting studies convening college and job training program instructors to determine the academic performance on NAEP reading and mathematics required for placement in entry-level colleges course or for enrollment in a job-training program.

This report describes the process and outcomes of a series of judgmental standard-setting (JSS) studies conducted to establish preparedness reference points on the 12th grade NAEP score scales in reading and mathematics. For each NAEP assessment, these points represent the academic performance required for placement in an entry-level, credit-bearing college course or for enrollment in a postsecondary job-training program.

JSS sessions were conducted from April through June 2011 and convened panels of college instructors and job training program instructors for five occupations to:

• Finalize the descriptions of minimal knowledge and skills that describe what students need to know and be able to do to be prepared for placement in a college course and/or a job-training course, and
• Determine the score on the NAEP scale that corresponds to the level of performance at the borderline (the cut score) and the percentage of students performing at or above the cut score.

The results of these studies include the descriptions of borderline performance, recommendations of cut scores, and recommendations of exemplar items illustrative of what students performing above the cut score know and can do.

Judgmental Standard Setting: JOB TRAINING | COLLEGE | READING | MATHEMATICS