Where do you find good multiple choice questions?

Yesterday I described my nefarious scheme for getting students to study for the trimester exam by means of a multiple choice extra credit exercise. I also indicated that the multiple choice portion of the AP trimester exam will include 23 multiple choice questions. During the trimester, I give multiple choice quizzes two-three times a week. And you will find out soon that, next month, my AP class will be plowing through many, many multiple choice exercises in preparation for the May AP exam.

You will find that I BELIEVE in the utility of a well-constructed multiple choice item to evaluate students’ understanding of physics concepts, and to help students confront their own misconceptions. Sure, many physics skills are better tested with free response items; I willingly concede that if the majority of your assessment is done with multiple choice, you get a biased account of a student’s physics ability. Too often, though, teachers and administrators dismiss multiple choice merely as the first and last resort of a lazy instructor.

Such is the pejorative connotation of multiple choice that when I was the first Woodberry teacher to acquire a scantron machine, I hid it in my office in order to avoid the inevitable soapboxing from my colleagues outside the science department. The machine is still in my office… but after eight years it’s become an open secret. At exam time I willingly help out the rebels from humanities departments who sneak down to use the machine.

Now, don’t think that I’m encouraging slack teaching. In order to be useful, a multiple choice question must be well-constructed. Writing good items is not a trivial exercise, as I’ve discovered numerous times. Not much is more embarrassing than going over a quiz in which the correct answer doesn’t appear in the choices, or in which the answers are not clearly different from one another. Initially, the front-end work necessary in finding or writing multiple choice questions cancels out the back-end work saved by grading via scantron machine.

(As an aside, my English department colleague El Molé invented the principle of conservation of exam workload – in writing an exam either you have to spend enormous time writing multiple choice or grading essays. The total time spent on the exam process is conserved regardless of how the exam is structured.)

Perhaps I’ve convinced you of the utility of multiple choice. Multiple choice practice might be useful and wonderful, but this post begs an obvious question – where in the heck do you find enough good multiple choice items for use in your class?

That’s a tough one, but I have a few suggestions. First of all, get good at evaluating the quality of an item. When you assign a question on a test or quiz, rewrite it immediately or throw it out if it didn’t work quite the way you thought. When you happen to see a good question somewhere, write it down before you forget.

The best source of multiple choice items is the College Board itself. A number of full-length AP exams have been released. Go attend an AP physics workshop, contact an AP physics consultant, or go to collegeboard.com and look for released exams. (Neither I nor anyone else is allowed to post content directly from an AP exam, as that would infringe on the AP program’s copyright and a plague of lawyers would descend upon me.) The College Board also writes the SAT II physics test, which consists of well-written and vetted multiple choice questions. Take a look at a sample test and use some of those problems.

I do not recommend most commercial AP or SAT II preparation books. It’s rather pathetic how out of touch most of these books are with the level or content of the exams, or sometimes even with what physics is all about. Similarly, I strongly recommend against fly-by-night companies such as the ubiquitous “D&L marketing” who send flyers peddling AP physics multiple choice tests.

Two books, though, are in fact useful. One is my own, 5 Steps to a 5: AP Physics B & C by Greg Jacobs and Josh Schulman. Yeah, I had better recommend my own book. One other good source is the older book published by Kaplan, written by Connie Wells and Hugh Henderson. Connie and Hugh are both AP readers, both former members of the Test Development Committee (the group that writes the AP test each year), and both should be on any list of the top 10 physics teachers in the USA. It’s worth finding a copy of the Henderson/Wells book – the newer Kaplan book has different authors, and I have not evaluated its quality.

Each year the American Association of Physics Teachers sponsors the Physics Bowl, a 40-question multiple choice contest. Old tests from 1994-2000 can be found at the PSRC website. Some questions are good, some aren’t, some won’t cover the topics you want; but one way or the other, Physics Bowl questions are an awesome resource.

The AAPT sells CDs of Physics Bowl tests and solutions from 2001-2007. They also sell a couple of other multiple choice tests on CD. These are worth the money.

If you’re looking for below-AP level multiple choice, a terrific source is the National Science League. Their contest is rather silly – my top 10 students in AP physics would have no excuse not to get a perfect score. But for my GENERAL physics class, the NSL test provides solid review questions. I’ve been buying this contest each year for a decade now, and so I have a large bank of basic questions for lower-level students.

Got a good source for multiple choice questions? Post a comment.

GCJ