Mailbag: Should you guess on multiple choice problems?

From Joey Konieczny, from Georgia:

Another question I had was about how you grade your multiple choice questions. The AP exam takes off 1/4 points for every wrong answer, correct? Do you do the same? Do you encourage your students to not answer your exam questions if they are not confident in their answer?

Yes, the AP multiple choice exam is graded exactly like the SAT: with 5 choices per item, students get 1 raw point per correct answer, and lose ¼ raw point per incorrect answer as a correction for guessing. And yes, I do the same on my in-class tests, because they are given as authentic AP practice exams.

However, I encourage my students to answer EVERY SINGLE multiple choice item, regardless of how confident they may be. Why? Look at the mathematics, to begin with.

Imagine a 100 question multiple choice test answered completely randomly, with absolutely no hint as to the correct answer. A person who leaves everything blank gets zero raw score. The person who answers every question will, on average, get 20 right for 20 raw points; but this person will get 80 wrong, for -20 raw points, leading to an overall raw score of: zero. Exactly the same as if the test were blank.

My point here is that, contrary to the myth that some idiot test-prep corporations propagate, there’s no harm in random guessing. On average, random guessing scores exactly the same as leaving blank. Of course, there’s really no point to truly random guessing: if someone runs out of time with ten questions left, I do not advise randomly filling in bubbles.

However, this analysis shows that there is, in fact, benefit to guessing that is in any way not random. Now, some will give the advice that if you can’t surely eliminate a couple of answers, then you shouldn’t guess. I disagree. My students – and, since you’re reading this blog, yours too – develop pretty good instincts about physics problems by the end of our year-long course. Even if they can’t bet their life on the wrongness of an answer choice, my students are certainly more than 20% likely to pick out the correct answer on virtually every multiple choice problem I’ve ever assigned. So they should always guess! In the best-case scenario, these non-random guesses add a few raw points to the overall raw score. And in the worst case – THIS IS THE CRUX OF MY ARGUMENT – these random guesses do no harm.

My rule is, if you read a question, mark an answer.

Now, it’s actually a rather tough process to convince students to guess, because they’ve heard for so long about guessing penalties. The most convincing argument to encourage students to guess on multiple choice questions came from my former student, Bret Holbrook, who truly understood the methods to my madness. He reminded his classmates that I make students do a correction for every multiple choice question that isn’t right, whether or not an incorrect answer was marked. So, he reasoned, he might as well guess. That gave him at least a 20% chance of avoiding the hard work inherent in the correction that he’d have to do anyway if he left the question blank.

You don't want homework to look like this --> .

I only assign about two problems per night, while math classes usually assign 10-30 problems per night. Yet, my physics assignment should take about the same amount of time to finish as a night of math homework. Not only are the physics questions more involved than their math counterparts, physics problems also require enormously more communication.

Without guidance from me, my students’ problem sets will look a lot like the picture to the right. Even with repeated written guidance, oral guidance, examples, threats, and groveling from me, it takes a long time to establish my expectations for problem presentation.

One trick that has helped tremendously is to require all homework to be done on UNLINED paper, with merely one or two problems per page. (One problem per page in AP physics; no more than two per page in general.) Notebook paper seems to constrain a response: all fractions are written on two lines, diagrams are rarely drawn; and if they are, they often fit nicely alongside the lines to the detriment of the communicative power of the diagram. Students schooled in resource conservation and basic economics put as many problems on a single page as they can fit.

A well presented problem usually includes three elements: diagrams, words, and mathematics. In fact, when I’m pressed for time, I will often grade problems cursorily by merely looking for these three elements and the answer. The blank canvas of the unlined paper, along with the mandate to fill up that canvas, seems to free up my students so they are more likely to include all three of the parts of carefully done homework.

It would seem that the cost associated with the benefit of homework on unlined paper would be the necessity for students to purchase expensive paper, along with the associated environmental cost of recycling or trashing the mounds of used homework assignments. I get around this issue by issuing biannual pleas to the school for usable unlined paper that would otherwise be dumped. Sure enough, my class’s needs have been more than met for a decade.

The library has given me their recycle pile, made up mainly of abandoned print jobs that were printed only on one side. (It’s obviously fine for homework to appear on the back of an old English essay, though students are initially hesitant not to present work on a perfectly clean sheet. What do I care about the flip side?) I make a point of keeping the recycling pile in the science department copy room neat, so I can just take it to my room for student use.

But perhaps the most awesome donations I’ve received come from the admissions and development offices. Whenever someone leaves, or whenever a telephone number changes, they must print new stationery. Their attics were full of stacks out-of-date sheets saying the equivalent of “From the desk of President Herbert Hoover.” The folks in these offices now make a point of sending old boxes of stationery to my room, where students love to use the high quality paper.

No one tries to fit lots of work into the corner of a page anymore. On the rare occasion at the beginning of the year when someone does so out of habit, I just remind him what he has to pay for really nice unlined paper.

GCJ

Mailbag: Dead Rats

I’m back from Yosemite, where I probably COULD have posted to this blog if I had wanted to – I saw not one but numerous people talking on cell phones while they hiked the trails. Nevertheless, I went somewhat old school, and stayed out of touch. I’ll be back to Woodberry on Tuesday, and frequent posts should begin again as the start of school approaches for me.

For now, chew on this question, asked by a participant in one of my AP Summer Institutes:

“I was re-reading some of the materials you shared, particularly your homework submission guidelines. There is one thing I don't get: What is a dead rat?”

-- Samuel Holiday, Pinson, Alabama

I learned the term a decade ago from Arkansas professor and AP reader Gay Stewart, though it wasn’t her invention, I don’t think.

Say you find a dead rat in a pickle barrel you're selling. Well, if you remove the dead rat before the customer sees it, you can still sell the pickle barrel, though possibly at a discount. If you don’t remove the dead rat, you're in big trouble.

In physics, a "dead rat" is a bloody ridiculous answer: a commercial airplane that has a mass of 10^2 kg... a car moving 10,000 m/s... not just an incorrect answer, but one that could and should be ruled out based on any kind of physical sense.

An unidentified dead rat causes a student big trouble to the tune of an enormous loss of points, regardless of the reason for the error. But if that student points out the dead rat and how he knows an answer is a dead rat, then he'll lose very little credit.

GCJ

AP physics – the first assignment

The last post discussed what goes on in my class on the first day of school. But what about the first night’s assignment?

I’ve not covered enough material by the end of day 1 that would justify assigning equilibrium problems. And, assigning math review is worse than useless – it’s harmful. I need problems for the first night that are doable without any in-class content, but that have useful pedagogical aims. (If I can’t justify homework as truly advancing students’ understanding, then I can’t assign it. There will be no busywork in my class.)

Consider what makes physics homework different from the homework that high school students are used to. A properly presented homework problem includes words, equations, diagrams, and a brief “comparison” in which the student shows an understanding of the physical meaning of a numerical answer. On the other hand, students habitually “solve” math problems by cramming a few numbers into the corner of a piece of notebook paper. The first day’s assignment should show clearly the level of thought, effort, and communication that will be regularly required.

So… here are two problems that I have assigned on the first night. The first is adapted nearly verbatim from a problem in Giancoli 5th edition.


1. An average family of four uses roughly 1200 liters – about 300 gallons – of water per day. How much depth would a lake lose per year if it uniformly covered an area of 50 square kilometers and supplied a local town with a population of 40,000 people? (Your comparison should discuss the size and/or depth of the lake compared to bodies of water you may be familiar with.)

2. Which is faster – your hair’s growth rate, or the speed of continental drift? (Your answer – in words with mathematical justification – is your comparison.)


Then, on day 2, here are two questions from the multiple choice quiz I give at the beginning of class. Note that the choices to the first question are local references… I encourage you to use this quiz with descriptions relevant to your class.

• How big is a 50 km2 lake?
(A) It would cover Woodberry and Orange, together
(B) About half the size of Woodberry’s campus
(C) About as big as the lakes on the golf course
(D) Would cover Woodberry and Charlottesville, together
(E) About one-tenth the size of one of the Great Lakes

• Which is faster, your hair’s growth rate, or the speed of continental drift?
(A) continental drift
(B) hair growth
(C) they essentially are the same


While everyone is working on these problems, I collect the homework. It is easy to flip through the papers to find all of the crazy answers to problem 1. I’ve had people tell me the lake’s depth dropped by as little as .8 mm, and as much as 87 million kilometers. As soon as I've gone over the quiz, I pick one egregiously silly depth and discuss why that can’t possibly be a correct answer. Thus, even before I graded anything, the homework problems have served a purpose: they’ve provoked a discussion of physical reasonability, and shown better than the best-written essay the difference between physics and math.

GCJ

First day of school -- DO PHYSICS

Think about the experience of high school students on the first day of school. They will likely attend four to six academic classes, each for somewhere between 40 and 90 minutes. What will happen in those classes?

Most teachers will take care of administrative minutia. Pass out and read the syllabus, hand out and sign for textbooks, go over rules of the class, how grades are assigned, and blah b-b-blah blah blah. Perhaps a few perceptive teachers might undertake a short discussion about the class’s overall goals, like “What was the most important event in American History?” But I would hazard that in most classes, the actual content covered on the first day is minimal, passive, and non-essential.

Physics can be different.

I teach juniors and seniors only, in general and in AP physics. Presumably the 16-18 year olds in my classes can read; so I send them the syllabus ahead of time via email, and make them read it. Presumably my upperclassmen have learned how to behave in a high school class; so I consider it unnecessary and condescending to discuss a list of class rules such as “respect one another” or “no chewing gum.” (How would YOU feel if you attended a conference which started with a litany of restrictive, prescriptive rules behind which is the underlying assumption that you will do all of these naughty things but for the recitation of said rules?)

Within fifteen minutes of my students’ arrival on the first day of AP, I dive into physics. We define a force as a push or a pull, measured on a scale; I write the definition of an object in equilibrium, and show how to solve equilibrium problems. By the end of the first day, the class is ready for the following quiz (which leads off day 2):

The box pictured above moves at constant speed to the left. Which of the following is correct?

(A) The situation is impossible. Since more forces act right, the block must move to the right.
(B) T3 > T1 + T2
(C) T3 < T2 + T2
(D) T3 = T1 + T2
(E) A relationship between the forces cannot be determined.

And then on day 2, I show with a quantitative demonstration how to deal with a force that acts at an angle. We’re off and running, such that the problems on the SECOND NIGHT OF CLASS are at the AP-level.

The same principle applies to general physics – on the very first day we are making position-time graphs with the motion detector, such that the second night’s problems can involve serious graphical kinematics.

And since most other teachers are talking about the penalties for late work while I’m holding an active class complete with demonstrations, I instantly capture attention. I do think that, in general, physics is more entertaining than most other subjects. But if nothing else, on day one I’ve made students FEEL like my class is special.