I spent last week at a PAWS range-finding event at the Pearson testing company's site in San Antonio. I worked with two other Wyoming teachers and two Pearson scorers to select the range of scores in 8th grade expressive and expository writing.
Here is the process we used:
1. We read and discussed the idea development portion of the expressive rubric.
2. We read somewhere around 15 papers aloud, scored them individually only on idea development, and then discussed the scores and came to consensus.
3. We read 13-15 more individually, then shared scores and came to consensus on each.
4. We followed this pattern for the traits of organization, voice, and conventions.
5. In the last step of this process, we scored roughly 30 more papers for all of the traits, using some specific papers from the initial scoring sets to help us maintain consistent levels of proficiency that we had set thoughout the first phases of the process.
6. We shifted to expository writing and followed each of the same steps through that process.
Yes, it was grueling, but I did gain a much clearer understanding of the bigger picture of this assessment, and I understand what happens once the papers are shipped off to Pearson. The two scoring room leaders who worked with my group were incredibly astute and conscientious in helping us work through this process. Without their knowledge, we probably wouldn't have maintained any sort of consistency.
I left feeling that the testing company does its job well; it scores accurately according to the criteria that is set before it. The problems come with all of the other variables - from the state department to the students. I hope we don't forget that this is only one small indicator of a student's ability to write.
