In conceptual physics, the second trimester exam covered:
* optics
* waves
* circuits
* motion
* force
* motion and force in two-dimensions
Three-quarters of the exam was on the latter three topics. Most of the questions were based on New York Regents exam questions. The exam included 40 multiple choice questions, along with 20 "justify your answer" items.
So which question out of 40 was the most frequently missed?
The reason I use the automated "Grademaster" sheets to score multiple choice is not merely convenience. When I've scanned all of my students' responses, I send through an "item analysis" sheet. I get statistics for the class's overall performance (i.e. the class average and a histogram of scores), along with a tally of the number of incorrect responses for each question.
I play predictive games as I'm running these multiple choice response sheets. I'm pretty good at predicting an individual student's score to within +/- 2 out of 40. I can also guess the overall class average to within a couple percent, every time.*
* In the television version of Jacobs Physics, the scanner is located in the teachers' lounge. The department -- played by actors far more attractive than real science teachers -- gathers round, with one guy taking bets. As each scoresheet is scanned, money changes hands, accompanied by hilarious quips. My character always ends up with a wad of cash, at least if my agent negotiates the TV rights correctly.**
** Before the humor-impaired come after me with pitchforks, this scene is entirely imaginary. I do not gamble on student performance. And the scanner is located just outside my office, not in the lounge.
In all seriousness, try playing this prediction game, especially with the overall average score. It's a way of keeping a teacher's expectations grounded in reality. If your prediction of the overall average score is well above the actual average -- say, by more than 8 percentage points -- then the students aren't understanding the material as well as you thought they were.* On the other hand, if you predict the class average too low, you have a different kind of disconnect. You can proceed with the course having increased confidence in your students' comprehension.
* Or, the test was more difficult than you thought, but I advise controlling test difficulty by using mostly items from an external source.
As soon as I'm done scanning, I look at the item analysis sheet to find out which problems were most often answered incorrectly. Consider it last-ditch proofreading. If three-quarters of the class miss a problem, either the problem was difficult... or I accidentally gridded the answer key wrong. That happens plenty of times. So I look at the two or three most-missed problems again, verifying that my answer is correct, possibly checking a few student sheets to see which choice they put down.
And that's how I've discovered a number of entrenched misconceptions. Usually, a follow-up quiz or homework addresses those misconceptions. But I have to see which mistakes the students actually make in order to follow up.
On this particular exam, I never would have guessed the most-missed problem. They nailed the elevator problem, they did fine with Newton's third law, they even did acceptably on the circuit questions, despite the three month lag between class coverage and exam.
Look at the picture at the top of the post, which is from a New York Regents exam. The problem asked for the direction of the wave's travel after reflecting from the barrier. Only 10 of 40 students answered correctly, even though I assumed that this law of reflection question would be a gimme. Most folks chose path D; the correct answer is C, because the incident and reflected angle have to be equal with respect to the normal. Why did they miss it? I assume because they quickly grabbed the answer with the 90 degree angle, rather than looking at the normal and the angles in play. When we return after break, I will include a question like this on a quiz.
Were the students missing the physics or were they mis-reading the diagram? Physics teachers look at the diagram and see that B is really the normal but in a world of "not to scale" diagrams I think it is awfully easy for a student to think angle of incidence is equal to angle of reflection and still pick D. Perhaps I'm being too easy on my students but I would put a normal line (separate from the answer choices) on the diagram.
ReplyDeleteWell, it's not a matter of "easy" or "hard", but of "right." It's clear that B is the normal line; and if you can't see it, you've got a protractor, so measure it. (Part of understanding reflection is to know that a normal is 90 degrees to the mirror.) Same thing with choosing between C and D: it's pretty obvious to me that C equalizes the incident and reflected angles, but if it's not obvious, measure it with the protractor. The whole point of this question is to determine who truly understands the law of reflection, what a "normal" means, what it looks like for an incident angle to equal a reflected angle. If you mis-read the diagram, you mis-understand the physics, too.
ReplyDeleteIf the students have a protractor available the test than I agree with you.
ReplyDelete