How CommonLit’s team makes valid, reliable assessments
Interim Assessments play a critical role in school districts. They are used to inform curriculum changes, identify student learning gaps, and to determine how resources can be used more effectively.
Since we place so much value in assessments, it’s important for districts to choose assessments that are well-constructed, valid, and reliable.
Recently our team talked to Jayne VanNosdall. She’s the Curriculum Supervisor of English for Brick Township, New Jersey. She told us that prior to utilizing CommonLit’s Interim Assessments, “The district had a home-grown 25 question test. However, it was pretty much rote and it wasn’t assessing the standards.” For Jayne and her team, it was a major problem because they “weren’t necessarily providing a challenge to the students, with regards to the type of texts or questions [they] were using.”
Jayne and her team are confident that CommonLit has had a really positive impact in their district. Jayne said, “with all of CommonLit’s tools, we’re better able to prepare students for our state test.”
We also had the chance to speak to Terri Bradshaw, the secondary literacy coach for Blount County, Tennessee. She told us, “The (CommonLit) assessments have been excellent. The questions are high-quality, and the way that the distractors are written are top notch as far as item writing goes. Also, the quality of the text, and the features in the text for the standards that you’re assessing are very clear.”
So, how do we make sure that our assessments are high quality? Below, I’m going to give a sneak peek into CommonLit’s thorough process of ensuring that students in our partner districts have access to Interim Assessments that are fair for all students, aligned to state standards, and valid indicators of students’ reading ability.
Curation
The assessment writing process begins with text curation. CommonLit’s literacy experts routinely start with roughly 100 texts per grade, before honing in on the 3 texts that will ultimately be used in the assessment. CommonLit’s interim assessments include 1 literary text and 2 paired informational texts. Texts are selected if they are grade-level appropriate and if a wide array of standards can be assessed using the passage. Texts are excluded from the assessment if they risk introducing racial, ethnic, economic, or gender bias. CommonLit is also careful to avoid texts that touch on sensitive topics that have the potential to elicit strong or negative reactions from students.
Question Drafting and Review
Once texts are chosen, CommonLit’s question writers begin drafting items. These questions are all standards-aligned, cover a wide range of standards, and focus on different parts of the text. Question writers also create a surplus of questions; as the review process continues, some of these questions will ultimately be eliminated.
After items are drafted, multiple trained senior reviewers analyze, recommend edits, and norm their edits through a collaborative process.
Field Testing and Final Edits
Passages and questions then undergo a field-testing process. CommonLit’s data analysts closely monitor the data collected. The team is on the lookout for questions that are too easy, too hard, or that yield inconsistent data. More specifically, the data analysts also flag the question distractors that are rarely selected and the question distractors that are frequently selected and may be too close to the correct answer.
Final Senior Editor Review
Based on the results of the field testing process, CommonLit’s senior reviewers then make final cuts. Revisions are made to distractors that were either too easy or too difficult and a final set of the most reliable questions are chosen for the Interim Assessment.
Conclusion
At CommonLit, we believe that schools and districts deserve access to the best assessments that can accurately measure students’ skills.
Click here to learn more about CommonLit’s District Solutions. To connect directly with me or another member of our School Partnerships Team, send an email to Partnerships@CommonLit.org
