Why four rather than five choices on AP Physics 1?

Blog- and AP- reader Barbara sends the question:

Any idea on the rationale for moving from five voices on the MC to four?

Barbara, mainly this was a reading density issue. 

Reading, writing, and understanding English* are inescabable and fundamental parts of learning physics.  Nevertheless, we want the language in questions to be straightforward and minimalist, such that the language doesn't become an obstacle to demonstrating physics knowledge.

* Or another language, of course... but the AP exam is in English. :-)

The College Board and ETS do psychometric** research investigating their exams, and their examination techniques.  For example, they've shown that deducting 1/4 of a point for an incorrect multiple choice answer doesn't differentiate between students any more than just scoring the number of correct answers directly.  At the AP reading, investigations have shown that grading a physics problem holistically*** produces scores indistinguishable from traditional grading.  

** I may have made that word up

*** Meaning something like "2 points for a complete answer, one for a partially complete answer, 0 for a lousy answer" as opposed to assigning each point to a specific element of the response

In terms of five vs. four multiple choice choices, data shows that either approach differentiates students of varying ability appropriately.  (I don't know, 'cause I never asked, whether five-choice questions differentiate better.  The statement I'm remembering is that four-choice produces statistically significant and reliable differentiation.)  

Once the case for the statistical validity of a four-choice exam was made, then it was a shoe-in as the superior option.  Statements from test developers suggested that question authors too often seemed stretched to create four incorrect choices that each made sense -- they got too many questions where some choices could be ruled out on the grounds of "this sounds totally silly and made up."  With only four choices, it's easier to create three incorrect yet plausible responses that directly test student misconceptions.

The bigger issue, though, was the reading burden on the student.  Even for a very well constructed five-choice item, the student still must take the time and intellectual effort to read an extra choice.  The psychometric studies suggested that most students were not, in fact, reading and understanding all five choices; and, that students who DID read all five choices often had to read them multiple times to make a reasonable decision as to the best answer.  

It was clear from the beginning of AP Physics 1 that this new exam would require considerably more verbal expression than AP Physics B did.  So the College Board and ETS made several changes to the format of the multiple choice, with the goal of minimizing the reading comprehension burden:

* Item authors are now required to justify the incorrect choices, explaining how each choice helps differentiate students who understand the physics targeted by the item from those who don't

* The multiple choice section has been reduced from 70 questions to 50 questions, giving students more time to digest the more involved language used in the new exam

* The "roman numeral" question type has been replaced by "multiple correct" items.  (You know, those questions that gave I, II, and III, and THEN gave lettered choice such as "I only" or "II and III, only".  The studies showed that the reading comprehension burden was especially high on these.  However, simply choosing the two out of four correct choices does not require significant additional reading over a standard question.)

* And, as we're discussing... the number of choices was reduced from 5 to 4.

Now that I've taught extensively under both four- and five-choice regimes, I do prefer the four-choice.  My observation is that on the occasional wordy conceptual problem, students can more often than before appropriately eliminate three incorrect choices in preference to identifying the correct answer directly.  I think -- based on no evidence but my own decades-honed instinct -- that with fewer choices the test does zoom in more sharply on my students' physics skills than if those students had to wade through and weigh one more option in every item.  If nothing else, I don't perceive the same level of mental fatigue after a practice test.  And that was kinda the whole goal.

GCJ