Disjointed thoughts on test construction

I'm giving the first test in my new "Honors Physics I" course, the course that's intended to foreshadow the future AP Physics I.  I've also been helping to write and prepare tests in conceptual and general (Regents) level physics.  Thus, I've been reflecting a bunch lately on methods of test construction.

Lyle Roelofs, who (perhaps tied with Walter Smith) was simply the best teaching physics professor ever, emphasized repeatedly to anyone who might teach physics: "The only time a teacher can be sure of a student's full attention is on a test.  So use tests to your advantage."  Thus the origin of my test corrections, the test-question-writing exercise, serious exam review,  and more.  But, if I expect my students to take the tests seriously as study tools, I have to take serious care in the construction of the test.

That care starts with a professional-looking test.  There's nothing wrong with handing out a nightly problem set via a sloppy email or via a handwritten slip of paper.  Practice multiple choice problems sometimes consist of faded xeroxes from 30-year-old master copies.  No problem, 'cause no one is expecting every night's problems to be beautiful.  However, a TEST should include clean, nicely-formatted proofread copies.  Mistakes should me not just minimized, but eliminated -- how do you justify docking a student's test grade for a lack of units when you yourself misprinted the units of acceleration?  Sure, stuff happens, but if more than one test in a year contains a major typo or a substantive error, you need to proofread better.

The format of each test, I think, should be generally consistent throughout the year.  Students are taught not to read directions on the SAT.  Why?  Because the directions for each section are the same on every test, every year; and because these directions are available ahead of time for preparation purposes.  A physics test is supposed to be a measure of a student's content knowledge.  Sure, careful reading of individual problems is essential to a student's successful performance... but we shouldn't surprise anyone with a different kind of question than they're used to.  

In an AP class, I give tests in a format identical to the AP exam:  Multiple choice, followed by free response.  In my general, honors, and conceptual classes, the format is always free response, short answer, multiple choice.  I hand out the instruction sheet and any reference tables before the test, so that students know what to expect.  

In Honors Physics I have control over test design, since we're not yet formally teaching to an AP test.  So I combine free response, short answer, and multiple choice into a single time period:  2 hours for the end-of-year cumulative, national exam, and 80 minutes for the monthly in-class tests.  The rule of thumb for timing:  about a minute and a half per mulitple choice item, about three minutes per short answer item.  For AP-style free response, give a bit longer than one minute per point -- for example, in an hour I expect students to be able to solve five 10-point problems, or two 15-pointers and two 10-pointers.  (As a comparison, the AP Physics B exam allows 90 minutes for 80 points of free response; the AP Physics C exams allow 45 minutes for 45 points of free response.)

If you're not teaching AP, consider switching the traditional order of the test.  I put the free response questions at the beginning of the test, and multiple choice at the end.  Why?  Because I've too often seen students get captivated by a one-point multiple choice question, leaving no time even to make a reasonable guess at the 15-point free response question.  If, on the other hand, someone gets hung up on a free response question, there might be time to make reasonable guesses at the multiple choice questions at the end.

The content of each test should be transparent, even if that means "everything we've ever covered."  I thoroughly approve of cumulative tests; why should I bother teaching in September if everyone's allowed to forget what we learned?  But I also approve of a clear course outline, indicating the general topics that have been covered in class and that will show up on the test.  A cumulative test is not a licence to play "gotcha!"  If you're consistent all year in what you expect students to understand in each unit, and if every test includes something from each previous unit, the class will recognize and meet your expectation that they learn physics for the long term.  The nice side effect is that final exam preparation becomes a piece of cake if all tests are cumulative.

Someone stopped me in the hall yesterday after the first test, and said, "Mr. Jacobs!  That was like an EXAM, not just a test!"  I smiled at him... imagine how seriously he'll take my actual trimester exam, now that he knows what my monthly tests are like.  And imagine how comfortable he'll be in May on a cumulative, year-long, national exam like the AP or the SAT II.

GCJ