The best teacher I've ever met, Matt Boesen, teaches US history and constitutional law. Debate tournaments have turned because his students can instantly remember and use facts from his classes. I have every intention of sitting in on the constitutional law class at some point in the future.
Importantly, Matt and I are very DISsimilar in some aspects of our teaching. The most glaring example is what a typical class looks like: Matt facilitates discussion while sitting with his students around an elliptical table, while (on the occasions when the students aren't themselves experimenting) I perform live demonstrations on a table elevated above rows of students. Partly that's a function of our personalities; primarily the difference is one of history vs. physics. Good physics teaching looks different from good history teaching. Not even all good physics teaching looks alike...
One point on which Matt and I actively disagree is the utility of multiple choice items on tests. His own take is that multiple choice is the creation of the devil, the lazy teacher's way of finding out which students are good test takers regardless of which students understand the course content. I certainly see his point within the teaching of history, where a better alternative to multiple choice is always available. Recall of facts can be tested more authentically with identification questions that don't give hints to the answer ("What were the provisions of the fugitive slave act?"). Higher-order thinking about history requires writing; not necessarily five-page essays, but paragraphs making connections between concepts ("Explain the political circumstances that led representatives from free states to support the fugitive slave act").
But physics is different. Well-designed multiple choice questions *can* authentically test understanding. The advantage then is that the short response time* allows a test to cover a broad swath of topics in a variety of contexts. Moreover, some physics questions are best phrased as a multiple choice item. For example, here's a question from my recent 9th grade conceptual test:
* since there's no writing necessary, you can reasonably give one question per two minutes or less
Ball A is dropped from rest and falls for 2 s. Ball B is also dropped from rest, but falls for 4 s. How far does ball B travel?
(A) one-fourth as far as ball A
(B) four times as far as ball A
(C) twice as far as ball A
(D) half as far as ball A
(E) the same distance as ball A
The incorrect answers are not chosen arbitrarily -- I could go through each incorrect choice to explain the misconception or mistake that would lead the student to choose that answer. In that sense, the item is an authentic test of reasoning with the equation d=(1/2)at2.
But this question doesn't work well at all without the choices! In response to just the prompt "How far does ball B travel," a student might try to plug in numbers to answer "19.60 m," in which case the question is evaluating a very different skill, the skill of plugging and chugging. Okay, so suppose I rewrite the stem to read "How much farther does ball A travel than ball B?" A reasonable answer: "14.7 m", which is the difference between ball A's 4.9 m and ball B's 19.6 m. Still, this is a different response than I wanted.
Rewriting again, maybe I say "How many times farther then ball A does ball B travel?" Or "What is the ratio of the distance ball B travels to the distance ball B travels?" Now the student's answer might be "4 meters." Aarrgh... you meant "ball B travels 4 times farther than ball A," but that's not what you said. Or, perhaps the student rounded weirdly to get "4.0816:1." That's not a physically useful answer. Grrr.
So on one hand I write this question as multiple choice because the choices frame the style of answer required. But it's more than that. The manner of the phrasing of the choices firmly defines the physical meaning of the answer. Any other phrasing I can think of can encourage a student to perform a mathematical manipulation. Upon doing test corrections, I don't want someone who missed this problem just to figure out how to do the math right to get "4"; the choices emphasize that the question asks about a performable experiment. I'm not likely to get an argument that "Look, if you play with the equations THIS way you get 2, like I did, so I should get credit." In the rare case someone tries to argue points with me, I don't engage; I just say, "okay, let's do the experiment."
Of course, like any teaching tool, multiple choice questions have to be used correctly in order to be useful. The questions must be well-written, so that it's highly likely that a correct answer comes not from gaming the test, but from good physics knowledge. Other forms of test questions must be used in concert with the multiple choice in order to get a complete picture of a student's ability. (An easy way to add variety and higher-order thinking to a test is to ask a multiple choice question, then say "Justify your answer.") And if you're going to test with multiple choice questions, students have to be used to seeing multiple choice questions on homework and quizzes; I'd say about half my general physics problem set questions are multiple choice with "justify your answer."
Next time colleagues or administrators challenge your use of multiple choice items, enthusiastically take them aside and show them a few well-designed physics questions which cry out for the multiple choice format. Show them this post. Go into a detailed discussion of the pedagogical philosophy of articulating physics concepts, not just solving math problems. Generally, you'll get one of two responses... virtually every non-physicist you attempt to engage in this discussion will (figuratively) run screaming rather than try to understand how physics teaching actually works.
Sometimes, though, you'll get the Boesens who enthusiastically listen, recognizing the differences among individuals and disciplines. These are the colleagues to treasure, because chances are you'll learn as much by listening to them as they learn by listening to you.
GCJ
What do you think of the F=ma exam as a source for good multiple-choice physics questions?
ReplyDeletehttp://www.aapt.org/physicsteam/2013/exams.cfm
This test, if you're not familiar with it, is the first qualifying contest for the national physics team sponsored by the American Association of Physics Teachers. So, by its very nature the test *must* be multiple choice format.
ReplyDeleteI loved most of the questions I read. This is an excellent, challenging contest. The questions are very difficult, at the high AP Physics C level. But in the context of my post above, are these good multiple-choice questions for a typical high school class? And if so, which ones?
The answer depends on the context of your class. If you're teaching AP Physics C, you must ask multiple choice questions in the style of the AP exam itself. You can find plenty of authentic questions on the College Board's released exams, so you might not need so many more. If you do, some of these are good, some aren't -- the linked AAPT contest allows calculator use, so questions have to be carefully vetted. It's a similar situation to the AAPT Physics Bowl, where every year about 6 of the 40 questions can be used for AP Physics B or AP Physics C.
In my post I'm discussing questions that I would ask as multiple choice items *even if I had no external forces driving the format of my tests*. This AAPT contest has many questions that are best phrased as multiple choice, but others that require some problem solving or explanation to be an ideal question. This is not to criticize the questions themselves, because of the national contest purpose of the test. As with virtually any source of material, the teacher has to do some critical analysis of the quality of the test items.
Hope that helps... I can discuss some items specifically if you'd like, but the answer usually boils down to "evaluate each item in the context of your course."
GCJ