Everything's a test -- there's no such thing as a "formative" or "summative" assessment

I don't think I had even heard the terms "formative assessment" and "summative assessment" until a year or two ago in connection with the new AP Physics 1 and 2 courses.*  If you're not familiar with them, the briefest definition I've discovered describes a summative assessment as an assessment of learning, while a formative assessment is an assessment for learning.   While I understand the meaning of the terms and their use in discussing how physics is taught, I have the deep-down suspicion that I'm hearing euphemisms reflecting some teachers' fundamental discomfort with the idea of a test.

* course materials written by education professors for education professors

Everything we do in our class is simultaneously a test and a learning opportunity.  The students -- and their teachers -- must get comfortable with the dual purpose of each activity.  In this sense, an academic class is no different from an athletic or artistic endeavor.  Basketball players are absolutely judged by the coach on their performance in practice and in games; nevertheless, unless the player's name is Iverson, both the game and practice provide opportunities for growth.  In the theatre, actors are judged by their peers and by the director in every rehearsal as well as in every performance; yet the best actors learn from both their successes and failures.

Each day, my students take a quiz and turn in a homework problem.  One might call these "formative assessments" because they are primarily for practice -- the students get constructive feedback about their knowledge and performance.  But I'm also acquiring information about my students' progress.  I need to know what they can do, both with collaboration and individually.  Both quizzes and homework count in my gradebook; the quizzes in particular often ask straight-up fact recall questions.  Thus, one would have to categorize each as "summative assessments" as well.  

Of course I give tests. I certainly call each one a "test," or even an "exam."  Tests are half of the term grade, with daily work counting the other half.  These two elements (75%) are combined with the trimester exam (25%) to get the overall trimester grade.  So my tests and exams are "summative assessments," right?  Wrong.  

Lyle Roelofs, the best college professor ever, explained that a test represents the only time when you can be positive that you have your students' full attention.  So, he suggested, make the students engage with the test questions that they get wrong in order to earn points back.  My students do test corrections, earning back half credit for anything they originally missed but then explain properly on the correction.  I've had colleagues complain that high-stakes testing is cruel and useless, that some exams should be replaced by more learning time.  My riposte: exam time is learning time.  It's the best learning time.  An exam used properly is the very definition of a "formative assessment."

To satisfy the sticklers among you, okay, each of my classes takes ONE "summative assessment" each year.  It's generally called the "AP Physics 1 Exam," or the "Conceptual Physics Final Exam."  These serious, high-stakes end-of-course exams are still tremendously useful teaching tools because they cause students to focus on a cumulative review that, without the high-stakes exam, would fall on deaf ears.  In other words, by the definitions above, the final exam is definitely "for learning," and could be categorized as formative.  So hah... my thesis stands.  Everything's a test, everything's a learning opportunity, and there's no point in making an artificial distinction between the formative and summative.

Mail Time: Struggling with the deeper content on the new AP 1 and 2 exams

Chris wrote in to ask for advice and guidance with the physics content on the new AP exams.  He has taught physics for a number of years, but he felt quite overwhelmed at his College Board workshop last week.  He feels unprepared to teach the new courses.  I have little doubt that I will be encountering a number of participants who feel similarly as I embark on my own summer institutes.  Here's my response:

Chris,

I hear you that you struggle with the deep nature of the content in AP Physics 1.  And I'm well aware that you're hardly alone.

You ask for links or lectures, but I have nothing magical.  The only way to get good at the deeper physics content is to engage with it the same way you ask your students to.  

As you design your course for next year, do the problems; present them to the level you'd expect your students to present them.  When you get stuck, ask someone you trust for help.  (That could be the consultant or a colleague from the workshop you just finished, or me, or a local college professor, or the AP teacher community on AP Central, or even an alumnus/a who has taken college physics.)  Pledge not to assign a problem this year that you haven't extensively worked through yourself.  This work doesn't have to be done in the summer: In my own first year teaching AP I spent essentially every fall Sunday at the bar watching NFL Sunday Ticket, while I wrote up solutions to the upcoming week's problems.  There's nothing wrong with being only a few days ahead of the students, as long as you're ahead.

Then get into your lab.  I'm a big fan of setting up the example problems you do in class (or the problems you assign for homework) as demonstrations or lab exercises.  Practice making measurements to experimentally verify the answer to a couple of problems that you've worked through.  You might find that you need some new equipment; get it before the school year starts, figure out where to borrow it, or put it on a wish list for when you are asked what money to spend.  Where you can't obtain new equipment, find alternate ways of making measurements.  Then when the school year starts, spend an hour each week in lab doing creative work figuring out how to verify predictions that you've made in class.  Nothing wrong with asking students to help out with this process -- late in the year especially, they might become better lab putterers than you are.

And finally, don't expect to be perfect when the school year starts.  Be honest with your students: you're learning along with them.  Don't be intimidated when you solve a problem incorrectly -- just figure out how to do it right, show the class, and move on.  Go ahead and assign a lab exercise that you've never done yourself, or one for which you only have a vague idea how to approach.  Put yourself in a lab group and work alongside your students.  When you become stumped in lab, again ask someone you trust for ideas.   Don't expect everything to succeed -- instead, just make good notes for next year about what worked and what didn't.

The goal should be that by the end (not the beginning) of the school year, you should be able to get a 5 on the AP Physics 1 exam.  Don't worry about what scores your students get.  In the second and third year, you can work on adding the cool teaching ideas that you discussed in your workshop or that you read online.

Point is, while you've definitely got a bunch of work ahead of you, don't discourage yourself by expecting to learn everything immediately.  Learning how to teach physics is a three year process.  Expect to feel somewhat inadequate leading into your first year -- I know I did.  But don't evaluate your progress until the third year is finished.  If at that point you still are having major content difficulties and your students are not passing, then it's time to find another line of work.  More likely, in May of 2017 you'll find that your students are doing well, you're confident in your understanding of the material, and you're excited to try out new teaching ideas that seemed ridiculous back when you took that June 2014 workshop.

Good luck.

Rewriting Physics B questions for Physics 2: 2014 B7, thin films

The new AP Physics 1 and 2 exams cover much of the same material that AP Physics B has covered for decades.  The style of questions will be completely different, though, requiring a tremendous amount of verbal response.  To help teachers understand how the new exam will look, the College Board has released 1.5 practice exams;* few other materials vetted by the committee are available.

* One official practice exam available to anyone who has submitted a course audit for AP Physics B or for the new courses; half a practice exam with the course description.

One of our task as AP Physics 1 and 2 teachers will be to develop a library of questions in the style of the new exams.  A great place to start is with the released Physics B exams.  But don't use them verbatim.  Rewrite the Physics B question such that calculation is minimized or eliminated; and such that students are asked to explain their understanding of each question.

As an example, take a look at problem 7 from the 2014 Physics B exam.  (You can find it at this link.)  The item presents a situation involving thin films, and then poses five different tasks.  Here is each, and how (or whether) it can be rewritten for the AP Physics 2 exam.

Part (a) is a straight-up determination of the frequency given the wavelength and speed of light.  Such straightforward "calculate this" questions will be vanishingly rare on the new exams.  I'd skip this part.

Part (b) asks for a calculation of the frequency of green light in the film; part (c) asks for a calculation of the wavelength in the film.  Think about what's important or interesting about these questions.  We're looking to see that the student recognizes that the frequency is unchanged in the film; then, we're looking to see the student use either λn = λ / n; or, we're looking to see the student use v = λf with a recalculated speed of light in the film.  So, to rewrite for physics 2, try asking a question that requires the student to know and articulate these issues.

I guess I'd make each of these parts a ranking task:  "Rank, from greatest to least, the three materials (air, oil, transparent plate) in order of the frequency of the green light in the material. If the green light has the same frequency in two or more materials, indicate so clearly in your ranking.  Justify your answer."  And again the same question for wavelength.  While the frequency question is pretty much know-it-or-don't, the justification of the wavelength ranking requires some good explanation.

Parts (d) and (e) are already exactly the kind of question that Physics 2 will be asking all over the place.*  Part (d) requires a justification that cannot be accomplished with simple reference to an equation; part (e) uses the prompts "describe" and "give an explanation for the phenomenon."  

* Although I'm told reliably that "place a check by your answer" format will be eliminated.  Part (d) would be phrased identically, but the "greater than, less than, or equal to" answer would be expected to be part of the student's prose..

If there's enough people who ask, I can write a series of suggestions about rephrasing old Physics B questions.  This approach -- determine the important conceptual element of the calculation, then ask a question that requires students to articulate that element -- should work to make any Physics B free response item into a Physics 1 or 2 problem.  

"Honors Physics" as an obsolete concept -- determining readiness for AP Physics 1 and 2

Vidhya writes in:

I was hoping that you would be able to provide some insight to a question I have about teaching the new AP Physics 1 course.  In my school students have previously taken Honors Physics before AP Physics. However, with the changes in the AP course, some feel that the information offered will be redundant and are thinking of just waiting until we offer AP Physics 2 the following year. In Honors Physics we cover Newtonian Mechanics – no electricity or rotational motion. Instead, I am seeing a lot of students who have never taken a physics course signing up for AP Physics 1.

In your opinion, is it better for students to have physics experience before taking Physics 1? Will students who are coming from Honors Physics where many of the topics are covered already be bored in Physics1? 

Hi, Vidhya!  This is one of the more common questions I've been asked about the redesigned AP courses.

Unlike Physics B, Physics 1 is in fact intended as a first year course.  Top students who have had a truly rigorous honors course are likely better off in AP Physics 2.  Topic coverage isn't what's important in that decision... it's purely the rigor of the honors course.  

Try giving some of these students the practice AP Physics 1 exam.  If they can't do it, then they really should do the AP Physics 1 course, because they'll be lost in physics 2.  If they can do fine on the non-rotational non-electricity questions, they should move into physics 2, where they will likely excel.

I'd suggest that an "Honors Physics" course is now essentially obsolete.  A student who is qualified for an honors course can do AP Physics 1 just fine as a first year course.  Then that student can go into physics 2. Many schools are replacing their honors physics - AP Physics B sequence by just AP Physics 1 - AP Physics 2.

Good luck!

GCJ

Designing your new course in the fall – don't make too-specific plans right now.

As you’re planning a new course – one you’ve never taught before – for the fall, it’s tempting to use the available summer time to plan everything down to the day.  While that sounds great in principle, such an attempt is doomed to disappointment.  You’ll never actually finish the planning this summer, for one thing.  More importantly, no battle plan survives contact with the enemy.  You might have the most wondrous sequence for October 17-24 to study linear momentum… but all it takes is a missed day for a statewide power outage on October 1 plus an unscheduled pep rally, and suddenly you have to re-plan.

Before you even begin, recognize that course development is a long-term, multi-year process.  It takes me two to five years before I’m truly comfortable with a set of course materials.  Expect that you'll prepare for activities, structures, and assignments that simply don’t work.  Be ready to switch gears midstream.*  Be like the electron, with a wave function well-distributed among many possibilities, but uncollapsed until the school  year progresses.

* Mixed metaphors are legal in physics blogs.

  

I start the summer with the same sort of planning I’d do for a class I’ve taught before:  I choose a plan for testing*, and choose major test dates that are not likely to conflict with other school events.  I make a general outline of topics overlaid on my school calendar, so I can see the approximate pace I need to set.  I write a brief syllabus communicating expectations for problem sets and such.

* For AP Physics 1, I think I’m going to try giving a weekly 10 question multiple choice test on Fridays, and a free response test every three or four weeks on lab day. 

Next, I prepare to write my tests.  This doesn't mean I write my tests ahead of time!  Since I’m choosing only the test dates in advance, and since every test is cumulative, I can’t necessarily predict what kinds of questions my class will be ready for.  So all I can do in the summer is collect and organize as broad a swath of test questions as possible.

Organization is the key here.  My own choice is to print a dedicated hard copy of every source of test questions that I might use.  I put this stack of paper in a folder, with each source labeled.  I also have a PDF scan of these sources in a computer file marked “test sources for AP Physics 1.”  When it comes time during the year to write a test, I browse through this large stack of hard copy to find questions on the correct topics.  As I use a question, I mark it off on the hard copy so that I avoid later duplication.  Then I go to the PDF source to copy and paste the question into Microsoft word.  In the summer, the goal isn't to write the tests – it’s to make the eventual process into a straightforward exercise in compilation rather than a creative writing job.

I write about two weeks’ worth of problem sets and in-class exercises.  For these early classes, I can be pretty accurate about the day to day progress of the course.  I’m as much brainstorming the style and scope of the questions I’m intending to ask as I am finding just the right questions to provide content practice.  I don't do any detailed planning beyond about that two week window, though.  

And finally, I gather resources related to each topic.  In particular, I've been recasting some of the quantitative demonstrations I've done for years as in-class predictive lab exercises; and I've been gathering equipment and brainstorming rotational motion labs and demonstrations.  Here I'm not making specific plans.  The day-to-day choices about the sequencing of each topic are, I think, best left for later. In the past when I have attempted to make a day-by-day plan months in advance, I've found in the moment that all that work was for naught.  Early on, I learn how the students react to my course structure and style.  I change a lot based on what works and what doesn't.  So it's best to wait to plan later units until I've had some experiences to build on.

Be warned: AP Physics 1 and 2 will be far, far more difficult to teach than you think.

I'm here at the reading grading problem 7 from the very last AP Physics B exam ever.  I've been discussing with other readers how this question provides a beautiful example of the difference between AP Physics B and the new AP Physics 1 and 2 exams.  More to the point, I think this problem can provide fair warning about just how unprepared the country's students are for the verbal reasoning required on the new exams.

Look at this thin films problem via the link above.  (I can't post it directly for lawerly reasons.)  It contains five sections.  The first three ask for calculations, of frequency in air, frequency in oil, and wavelength in oil.  These are the classic AP Physics B - style questions, in which mathematical reasoning with reference to formulae is prized.  On similar problems in the past, liberal partial credit has been awarded to those who made any sort of reasonable attempt to approach the calculations.  Questions in this style -- and the liberal, somewhat-gameable partial credit -- will not exist in AP Physics 1 and 2.

The last two parts to question 7, though... they are worthy of the new exams.  

Part (d) asks about the index of refraction of one of the substances.  Previous thin films questions have asked students to calculate the minimum thickness of the film; knowing how many phase changes happen, or how the phase changes might affect the result, was worth one measly point at most.  But this particular question can't be gamed in any way by referring to an equation, relevant or not.  The very nature of the question requires the student to explain that two phase changes are necessary in order to produce constructive interference; and that the only way to get a second phase change is for the bottom substance's n to be greater than the oil's n, just as the oil's n is greater than that of air.  The students themselves must generate the discussion of phase changes and interference properties, not merely take the question's cue and apply a formula.

Similarly, part (e) asks about an observation of the thin film from a different angle than straight in.  Oops -- again, no formula exists to be applied.  The only possible approach is for the student to explain -- unprompted, with no hints available from the question stem or formula sheet -- that the extra path length traveled in the oil determines which wavelength interferes constructively.  And since that path length is longer at the wider angle, the observed wavelength becomes longer, i.e. more red.  

The new exams will include almost exclusively these unguided demands for description of underlying physics principles.  

Currently, smart students who have been taught that physics can be gamed by plugging into formulas can earn 3s on the AP exam.  That's gonna change.  A plug and chug student can't answer these descriptive questions.  Trust me.  I've seen such questions (and the lack of reasonable answers from the vast majority of students) for years.  With few if any "calculate" questions to fall back on, there's gonna be wailing and gnashing of teeth around the country.  Not just by students, either... physics teachers who don't want to change their approach will be caught out.  They'll complain about the new exam, finding reasons that it's unfair or wrong.  

Oh well.  It is incumbent on physics teachers to start working now on changing our approach to our classes.  Don't give credit on a homework or quiz question for bare calculation.  De-emphasize calculation -- make every assignment require writing in sentences.  Do experiments, and make the students explain their experimental methods and results on paper.  Don't reward correct answers unless those answers are supported by correct reasoning.

And when your students complain, just show them questions like number 7 on the 2014 exam.  It's not *your* fault that they have to explain their reasoning more thoroughly on the new exams.  It's that evil College Board.  Hide behind the monolithic corporation.  But whether you or your students like it or not, the times, they are a' changin'.