Partnership for Assessment of Readiness for College and Careers | PARCC
Partnership for Assessment of Readiness for College and Careers
The PARCC states have carried out research to establish the validity of the assessments. Below are descriptions of the completed research reports. In addition, recent independent studies have found PARCC aligns to the standards and assessment criteria that matter most.

researchrequest

Related to Performance Level Setting

PARCC applied an evidence-based standard setting approach that used empirical data along with expert judgment to determine threshold scores to define the five adopted performance levels. 

Performance Level Setting Technical Report. (December 2015) The technical report provides information about the evidence-based standard setting process PARCC used to set threshold scores for each performance level in summer 2015.

PARCC Benchmarking Study. This study gathered external information that provides estimates of the percentage of high school students graduating college- and career-ready as well as the percentage of students on track for readiness in earlier grades. The PARCC assessments and Level 4 expectations were compared with international, national, and legacy state assessments. Specific external comparisons included PISA, NAEP, ACT, and SAT.

Postsecondary Educators Judgment Study. The purpose of this study was to provide information to support the Evidence Based Standard Setting (EBSS) process. Professors and instructors of entry-level college credit bearing courses from institutions of higher education in PARCC states provided recommendations on the minimum level of performance on PARCC items that would indicate students were 'academically ready' to take and succeed in postsecondary courses. 

Field Test Studies

The following studies were conducted with data from the PARCC 2014 spring field tests.   The studies were conducted to inform decisions about the development, implementation and scoring of future operational assessments and to gather validity evidence for the assessment program. Read a summary of the key lessons learned from the field test.

Quality of Test Administration. The purpose of this study was to investigate the quality of the administration of the spring 2014 field test. This report identifies issues in test administration, offers suggestions for addressing those issues, and describes actions that PARCC states have implemented in response.

Device Comparability Study. This study examined whether student performance on computer-based tests is affected by the type of device used (computer vs. tablet).  The study included students in grades 4 and 8 in ELA/L and math, and grade 10 ELA/L and Geometry students.

Mode Comparability Study. This study was conducted to determine 1) whether items/tasks perform similarly across paper and computer-based tests, and 2) whether student performance is similar on computer-based tests and paper-based tests.

Accessibility Study. The PARCC accessibility study examined whether assessments are accessible for all students including students with disabilities and English learners, and free of potential bias.  Students’ understanding of the format of the assessments and the use the technology was also examined. 

Feasibility of International Benchmarking Study. This study examined the degree to which PARCC and the international assessments measure the same or similar content.

Comparability of High School Mathematics End of Course Assessments Study. The study investigated the extent to which PARCC items administered to students in the traditional course sequence (Algebra 1, Geometry, Algebra 2) and the integrated course sequence (Integrated Mathematics 1, 2, 3) can be placed on the same scale to support comparisons of student achievement in the respective course sequences.

Cognitive Complexity Study. The study investigated the degree to which item-level cognitive complexity measures can predict item difficulty. The study also provides an evaluation of the use of cognitive complexity measures in item development, and the implementation of the cognitive complexity framework.

Quality of Items, Tasks and Stimuli Study. This study was designed to assess the psychometric quality of the content of PARCC field test forms. The study also examined whether students interact with items/tasks as intended, whether they were given enough time to complete the assessments, and the degree to which PARCC scoring rubrics allow accurate and reliable scoring.

Score Report Design Study. The study was designed to capture the priorities of PARCC stakeholders and to combine them with psychometric expertise in reporting best practices in usability research. Over a course of several months, different report design options were created and evaluated. The study report provides recommendations for the design of student-level and aggregate score reports based on evidence gathered during the study.

Automated Scoring Proof of Concept Study. The purpose of the study was to evaluate whether machine scoring can be used in scoring of Prose Constructed Response (PCR) in PARCC ELA/Literacy assessments.  

References and Suggested Readings Regarding Automated ScoringView the citation for the Automated Scoring Proof of Concept Study, along with a list of documents referenced in the report.

Operational Studies

College and Career Readiness Benchmark Validation Study. (March 2017) This two-phase study examines high school assessments’ college and career readiness or on-track benchmarks. In the first phase – completed in 2017 – performance data from the PARCC high school assessments from 2014-2015 and 2015-2016 was used to examine the relationships between PARCC’s level 4 benchmark and the college readiness benchmarks established by the College Board (PSAT and SAT) and ACT. The second phase - to be completed in 2018 – will involve gathering longitudinal data about academic performance in entry-level college credit-bearing courses for students who took PARCC assessments during high school.

Automated Scoring Study.
(March 2016) Building upon the encouraging results of the Automated Scoring Proof of Concept study conducted in 2014 using data from the PARCC field test, Phase 1 of this study used operational data from spring 2015 to examine the viability of a combined human and automated approach to enhance scoring efficiency. Phase 2 of this study extended the Phase 1 research by focusing on investigations related to the implementation of automated scoring for PARCC essays in spring 2016.

Device Comparability. (February 2016) This report describes a study of the comparability of PARCC assessments administered on computers (e.g., laptops, desktops, notebooks) and tablets. The goal of this study was to investigate whether overall test performance was similar for students testing on computers and tablets. This study examined performance on eight PARCC assessments: grade 5 math, grade 7 math, Algebra 1, Geometry, Algebra 2, grade 3 ELA, grade 7 ELA, and grade 9 ELA.

Quality of Test Administration. (October 2015) The purpose of this study was to investigate the quality of the administration of the PARCC assessment during its first operational year (2014 – 2015). The report documents improvements from the Field Test administration and identifies additional recommendations for continued enhancement.

Accessibility of New Items. (November 2015) This study examines the usability of a tool that allows students to draw responses to select mathematics tasks on PARCC’s computer-based assessments. The first component tested the usability of the drawing tool interaction on the PARCC mathematics assessment with students in grades 3, 9, 10, and 11. The second component of the study focused on how students with disabilities were able to interact with the drawing tool. The third component compared performance on grade 3 PARCC mathematics field-test items taken with and without a drawing tool. Further studies with the drawing tool are in progress.

Early Post-Equating Sampling. (March 2016) The study focuses on the psychometric considerations for conducting early post equating. Early post equating is a technical process conducted to establish comparable scores on different versions of a test using a sample of students – who have tested early in the administration window – to match relevant characteristics of the full student population. This allows score reporting to occur earlier than if equating was based on all students testing throughout the administration window.

Technical Reports

2016 Technical Report. (January 2017) The technical report provides descriptive information about the design and implementation of the PARCC assessments, as well as information related to the statistical properties of the tests for interested readers. The report describes the 2015-2016 administration of the PARCC assessments and includes the following topics:


  • Background and purpose of the assessments
  • Test development of items and forms
  • Test administration, security, and scoring
  • Test taker characteristics
  • Classical item analyses and differential item functioning 
  • Reliability and validity of scores
  • Item response theory (IRT) calibration and scaling
  • Performance level setting
  • Development of the score reporting scales and student performance
  • Quality control procedures

2015 Technical Report. (April 2016) The technical report provides descriptive information about the design and implementation of the PARCC assessments, as well as information related to the statistical properties of the tests for interested readers. The report describes the 2014-2015 administration of the PARCC assessments.

2014 PARCC Field Test Technical Report. (June 2015) The technical report provides descriptive information about the design and implementation of the PARCC assessments, as well as information related to the statistical properties of the tests for interested readers. The report describes the spring 2014 field test administration of the PARCC assessments and includes the following topics:

  • Background and purpose of the assessments
  • Test design and Field Test form development
  • Test administration, security, and scoring
  • Field test samples
  • Classical item analyses and differential item functioning
  • Reliability and validity of scores
  • Selection of Item Response Theory (IRT) models and calibration
  • Analyses to inform operational scores
  • Quality control procedures
  • Limitations and recommendations


White Papers

PARCC commissioned several white papers discussing psychometric and assessment design and administration issues. PDFs are available below:

 

  Contact PARCC’s research team for more information about the PARCC states' research and evaluation projects. 


Required

Required

Required

Required

Required

Invalid Input

Stay Informed:

Please enter a valid email address.

Connect:

connect twitterconnect facebookconnect youtubeconnect linkedin