Posts
At the AP Physics Reading we grade over 100,000 exams over just seven days. While accuracy is our primary concern, we cannot neglect the importance of speed. We all have plane tickets home at the end of the week. We don’t have a choice: finish, or else.[1]
The most important way we speed the reading is to have each reader grade the same problem throughout the week. After a few days, the rubric for that problem becomes instinctive. We begin to see the correct answers, and the common wrong answers, in our dreams at night. It seems as if our brains become hard-wired – assigning a score to each part takes place without conscious thought.
In one sense, then, grading speed depends most on the talent and motivation of each individual reader. We share our personal tips with each other because we’re all nerdly professionals. Only a physics teacher would be proud of discovering a shortcut that might save two seconds per exam. Competitive drive plays a role in speeding the reading, as well: though it is drilled into our heads that accuracy is paramount, in every room is a chart on which readers tally the packs they grade. Everyone, including table leaders, can see who’s fast and who’s slow. A reader who feels a bit lazy as the afternoon drags on might be tempted to slack; but if he grades half as many tests as everyone else, his slackage will be apparent to all. On the other hand, we can all fete the speedsters and ask them for helpful advice.
With so many readers and problems to grade, seemingly inconsequential time saving methods can be disproportionately effective. If there’s a way to help each reader grade just a few more problems every hour, that could extrapolate to an improvement the order of thousands of exams every day of the reading. So, you ask, what are some reading “mechanics” that save itty-bits of time here and there?
Back when I began grading in 1999, grades were recorded directly on each exam in felt-tip pen. We would pick up a folder containing 25 exams. Aides had already turned each exam to the correct problem. We would place the pack of exams on a cardboard ramp inclined about 5-10 degrees, allowing us to pull down the next exam as soon as we marked the first. When a pack finished, we put it in the “out” box near the door, picked up a new pack, and did it all again.
This grading style was very smooth for the readers, but required intense work from the reading aides. Since we wrote directly on the exam, it was important that the reader from problem 3 couldn’t see the scores from problems 1 and 2. To avoid subconscious prejudice based on a student’s previous performance, aides covered scores with opaque tape. After all problems were graded, aides removed the tape, and entered the scores into a computer. Even once each individual exam was graded, significant work remained before grades could be released to students and colleges.
About five years ago, physics started using a quicker and better scoring system. We still pick up a folder of 25 exams, but these are no longer turned to the correct problem. The exams are carefully kept in order; we turn an exam to our problem, grade it, and move on to the next exam. Rather than writing the scores on the exams, instead we bubble in the scores with a number 2 pencil on a separate sheet of paper. When a pack of tests is graded, aides just take the scoresheet to the “bubble room,” where it’s run through a machine. Thus, data entry is automatic. We can see instantly score distributions, how the free response scores correlate to students’ multiple choice, whether individual readers are too lenient or too strict… all is known as soon as the bubble sheets are scanned.
What relevance, you ask, might the AP reading process have to a regular high school classroom? After all, we don’t have a sophisticated data entry system, nor do we have an army of aides. Nevertheless, I have tried to take some ideas from the reading to move my nightly grading along. Primarily, I’ve tried ditching a traditional gradebook entirely. Instead, I print sheets of paper with a class list, and put these sheets inside the pockets of the folders that hold my assignments. When it’s time to grade, I just take out the folder. I write the grades directly on the sheet of paper. When I return the assignment to the students, I leave the completed grading sheet in the folder. By the end of the marking period, I have accumulated a stack of these sheets, which I enter into the computer.
What time does that save? I don’t have to carry or organize a grade book. When I sit down to grade, I have to grab one fewer thing. A silly benefit is that it becomes easier to convince myself to sit down to grade a stack of homework: since I don’t have to futz with the gradebook, since I can just start assigning and recording scores, I tend to get started more willingly.
Now, I’m not suggesting that using loose paper rather than a gradebook is going to work for you. I’m merely offering the idea that some thought towards logistics might allow you to save little bits of time this coming school year. And, those little bits add up.
GCJ
[1] Or else what? It’s never been entirely clear. Fortunately, in the ten years I’ve read we’ve only once even had to work all the way to the end of the last day. The nine other readings have concluded with hours or even days to spare.
The most important way we speed the reading is to have each reader grade the same problem throughout the week. After a few days, the rubric for that problem becomes instinctive. We begin to see the correct answers, and the common wrong answers, in our dreams at night. It seems as if our brains become hard-wired – assigning a score to each part takes place without conscious thought.
In one sense, then, grading speed depends most on the talent and motivation of each individual reader. We share our personal tips with each other because we’re all nerdly professionals. Only a physics teacher would be proud of discovering a shortcut that might save two seconds per exam. Competitive drive plays a role in speeding the reading, as well: though it is drilled into our heads that accuracy is paramount, in every room is a chart on which readers tally the packs they grade. Everyone, including table leaders, can see who’s fast and who’s slow. A reader who feels a bit lazy as the afternoon drags on might be tempted to slack; but if he grades half as many tests as everyone else, his slackage will be apparent to all. On the other hand, we can all fete the speedsters and ask them for helpful advice.
With so many readers and problems to grade, seemingly inconsequential time saving methods can be disproportionately effective. If there’s a way to help each reader grade just a few more problems every hour, that could extrapolate to an improvement the order of thousands of exams every day of the reading. So, you ask, what are some reading “mechanics” that save itty-bits of time here and there?
Back when I began grading in 1999, grades were recorded directly on each exam in felt-tip pen. We would pick up a folder containing 25 exams. Aides had already turned each exam to the correct problem. We would place the pack of exams on a cardboard ramp inclined about 5-10 degrees, allowing us to pull down the next exam as soon as we marked the first. When a pack finished, we put it in the “out” box near the door, picked up a new pack, and did it all again.
This grading style was very smooth for the readers, but required intense work from the reading aides. Since we wrote directly on the exam, it was important that the reader from problem 3 couldn’t see the scores from problems 1 and 2. To avoid subconscious prejudice based on a student’s previous performance, aides covered scores with opaque tape. After all problems were graded, aides removed the tape, and entered the scores into a computer. Even once each individual exam was graded, significant work remained before grades could be released to students and colleges.
About five years ago, physics started using a quicker and better scoring system. We still pick up a folder of 25 exams, but these are no longer turned to the correct problem. The exams are carefully kept in order; we turn an exam to our problem, grade it, and move on to the next exam. Rather than writing the scores on the exams, instead we bubble in the scores with a number 2 pencil on a separate sheet of paper. When a pack of tests is graded, aides just take the scoresheet to the “bubble room,” where it’s run through a machine. Thus, data entry is automatic. We can see instantly score distributions, how the free response scores correlate to students’ multiple choice, whether individual readers are too lenient or too strict… all is known as soon as the bubble sheets are scanned.
What relevance, you ask, might the AP reading process have to a regular high school classroom? After all, we don’t have a sophisticated data entry system, nor do we have an army of aides. Nevertheless, I have tried to take some ideas from the reading to move my nightly grading along. Primarily, I’ve tried ditching a traditional gradebook entirely. Instead, I print sheets of paper with a class list, and put these sheets inside the pockets of the folders that hold my assignments. When it’s time to grade, I just take out the folder. I write the grades directly on the sheet of paper. When I return the assignment to the students, I leave the completed grading sheet in the folder. By the end of the marking period, I have accumulated a stack of these sheets, which I enter into the computer.
What time does that save? I don’t have to carry or organize a grade book. When I sit down to grade, I have to grab one fewer thing. A silly benefit is that it becomes easier to convince myself to sit down to grade a stack of homework: since I don’t have to futz with the gradebook, since I can just start assigning and recording scores, I tend to get started more willingly.
Now, I’m not suggesting that using loose paper rather than a gradebook is going to work for you. I’m merely offering the idea that some thought towards logistics might allow you to save little bits of time this coming school year. And, those little bits add up.
GCJ
[1] Or else what? It’s never been entirely clear. Fortunately, in the ten years I’ve read we’ve only once even had to work all the way to the end of the last day. The nine other readings have concluded with hours or even days to spare.
I just saw this, but I wanted to share:
ReplyDeleteLike the AP reading, I grade one test question for ALL students. Then I start at the top of the stack and grade the next question. It helps with consistency because I'm grading the same question repeatedly and makes the process pretty anonymous (except for easily recognizable handwriting). I'm not influenced by scores on other questions and I move through the grading faster.
I write the total points on each question with a fairly large, easily-found number. I have a spreadsheet with the students' name on the top and question number on the side. Adding up their scores is a simple matter of flipping through each test and entering numbers with the number pad, using a sum function at the end. (If you were going to do it in your head or type it into a calculator, anyways why not make it a more permanent record?) I don't alphabetize stacks of tests: I type the student's name into the spreadsheet as I pick up a test from the pile. Excel can alphabetize the spreadsheet for gradebook purposes. I don't even write the total score on the test, the students can add the points themselves and compare them with the total on the gradebook. (This works best when the test already says how many points each question is worth. If I don't have that information, I'll write it on the board when they get the test back.)
Like the "bubble room", my spreadsheet becomes a gold mine of data to sift through. With a few formulas and conditional color formatting, I can immediately see how each student did on the test and how the class as a whole performed on each question. I can better target my reteachig to the class and identify the students who need one-on-one help. This is probably the best part about the process. I acquire all this information without any real extra work on my part and it makes grading consistent and relatively quick.