What does a 5 on an AP Physics 1 exam mean? It still means an A, but read on...

Back in July, I posted about the results of the first ever AP Physics 1 exam.  Executive summary: The percentage of students earning each grade dropped.  But since the pool of students taking the exam nearly doubled, more students than ever "passed" the algebra-based AP physics exam.

The comment section of that post has been active and interesting.  One comment in particular deserves an extended response.  Aaron Shoolroy points out a seeming disconnect in the College Board's statements.

On one hand, Aaron saw that university physics professors -- the ones who are ultimately responsible for awarding credit for AP exams -- were "pleased at the depth of knowledge that the test assessed."  That's correct.  The College Board has done an excellent job communicating with physics departments, explaining why this exam is harder, showing them that students who do well on AP Physics 1 really, really know their stuff.

On the other hand, though, Aaron points out that the College Board has marketed the AP program as a college equivalent.  That is, they claim that a student who earns a 5 on the AP exam would have earned an A in the college course.  "Not a chance," to paraphrase Aaron, and others who have made similar comments.  "No professors are only giving 4% of their students As.  I know my students will be doing much better in college physics than they are doing on this exam.  What gives?"

What gives is a subtle shift in philosophy from the College Board.

For decades, the College Board has aimed their AP exams at the typical introductory university course.  They have done detailed statistical analysis to demonstrate that AP exams match their college equivalent in content, in skill evaluation, and in student performance.  That analysis included -- as a matter of College Board policy -- a vast cross-section of institutions of higher education, all the way from community college to state school to ivy.  All introductory college physics courses are different; previously, the AP exam was aimed dead-center.

That philosophy has changed with the redesigned science courses.  Now the College Board designs AP courses to be equivalent to the "best" college courses.  The increased emphasis on skill development over content means that AP exams evaluate skills taught with the "best practices" of science teaching.

When the cutoff scores for a 5, 4, 3, 2, 1 were set last year, the College Board sought input from high school teachers and college professors about what standard they would expect their own students to attain on each of the exam problems to earn each grade.  They used statistical information from pilot exams in some of these "best practices" courses to correlate scores to grades.  They did NOT start with "what percentage of students should earn 5s or As?"  

So Aaron, you're right -- most professors are not giving only 4% of their students As; and many of our students who earned 3s and 4s on AP Physics 1 will go on to earn As in college physics.  But neither of those facts are relevant to the score setting process.  The relevant question is, given this exam, what level of performance would earn a student an A in the best college courses in the country?  

I take no position as to whether the College Board's new philosophy is right, or good, or good for the general state of physics education, or good for your or my students.  I have some opinions, both positive and negative, that don't belong in this post.  Today I'm simply explaining the philosophy behind the AP score setting.

What I can say is that the College Board's new philosophy is internally consistent, and that the score cutoffs were derived fairly and openly based on the criteria developed by the redesign committees and the College Board's executives.

Demo: How can something move down if I'm pulling it up?

 

The picture above is one frame of a video from class yesterday.  Hanging from the string is a PASCO cart with the "visual accelerometer" attachment.  Before this demonstration, I spent some time convincing the class that the lights on the cart correspond to the direction of acceleration.  Thus, everyone is well convinced in this screenshot that the cart has an upward acceleration.

We've used Vernier force probes before, too.  I showed the class that the cart weighs 7 N.   They can immediately see in the screenshot that the force probe reading, which is also the force of the string on the cart, is more like 11 N.  They're well convinced now, experimentally, that the upward force of the string on the cart is greater than the downward force of the earth on the cart.  They know that the net force must be 4 N, upward.  

The class is, unfortunately, also well convinced that the cart is moving upward.  Oops.

"BOUX," I've said for years.  The net force is not in the direction of motion -- the net force is in the direction of acceleration.  And we just spent a whole unit discussing how acceleration is in the opposite direction of motion when something slows down.  This cart could be moving down but slowing down.

For the first time, though, I can give immediate, undeniable experimental evidence.  I can just run the video forward for a few frames -- the class sees that the cart was, undeniably, moving downward.  Even though the acceleration and the net force was upward.  Cool.

Hints:  This was one of my early attempts at obtaining this kind of video.  I discovered I needed to use the "slo-mo" function on my iphone 6.  Try to lower (or raise) the cart such that the string never goes slack -- I didn't do that well in this trial.  

There is, in fact, a couple frames of lag between the appearance of the green dots and the force graph displaying greater than 7 N.  That's the computer processing and displaying the output from the probe, which is not instantaneous.  You can even see screen flicker in the video, since I'm recording at ~100 fps while the screen refreshes at 60 fps.  But the video is still convincing.

And finally, you can certainly get the students themselves to make this video.  Tell 'em to pull out their phones and record.  Chances are that at least one student gets a good recording.  Then have that student show a screenshot like the one above to classmates, explaining carefully the relevance of the green dots, the relevance of the force probe reading, and how the video shows the cart moving down, not up.