Paper - A Study of Irrelevant Items - Impact on Bookmark Placement and Implications for College and Career Readiness

A Study of “Irrelevant” Items: Impact on Bookmark Placement and Implications for College and Career Readiness

NAGB, 2012, GRADE 12

This special study analyzes the issue of irrelevant items and the potential impact of these items on panelists' judgements in the Judgmental Standard Setting (JSS) studies to identify NAEP cut scores for academic preparedness for college and job training.

The Judgmental Standard Setting (JSS) studies conducted for 12th Grade NAEP Preparedness Research are the focus of this paper. This was the first effort to set a cut score on NAEP to represent minimal academic preparedness for both college course placement and entry to job-training programs. Although the results of these studies did not lead to points to be reported on the NAEP scale to serve as indicators of academic preparedness in the job-training programs, valuable information was generated through the studies. The results of these studies point to important differences between the knowledge, skills, and abilities needed for entry in these job-training programs and those needed for college course work.

This paper reports on the issues surrounding the items deemed irrelevant by the panels of college and job training instructors convened for the JSS studies. This special study was designed and implemented to collect more information about the issue of irrelevant items and the potential impact of these items on the judgments of panelists.

Judgmental Standard Setting: JOB TRAINING | PLACEMENT | COLLEGE | READING | MATHEMATICS