11 July 2015

AP Physics 1 scores 2015 -- more people passed Physics 1 than Physics B, and other commentary.

By now those of you who taught the inaugural year of AP Physics 1 have seen how their students did.  Not like the old physics B, eh?  Let's talk about the reasons for the ostensible precipitous decline in the scores.

Firstly, the raw score necessary to earn each AP grade has increased, by about 5-6% across the board.  Trinna Johnson and Trevor Packer sent a letter to the "AP Teacher Community" discussion group describing the score-setting process in tremendous detail.  In that letter, they revealed the grade cutoffs, which I've converted to percentage of available points necessary for each grade:

AP PHYSICS 1 GRADE          Percentage of available points on the test
     5                                                       71%
     4                                                       55%
     3                                                       41%
     2                                                       26%

The old physics B exam, typically, had cutoff scores of 65%-50%-35%-25%.  It takes more correct answers to pass now.

The AP Physics 1 exam, though, is considerably more difficult that Physics B.  There are no pity points available for simple calculations.  Synthesis is prized over recall.  There's no room to hide -- the questions probe for explanations rather than answers.  Due to the higher raw score cutoffs, we would expect fewer of our students to pass even on an exam of equivalent difficulty to AP Physics B. Now we have two effects that combine to reduce overall exam grades: a harder exam AND higher cutoff scores.

And finally, consider the population of students who took AP Physics 1 this year.

In 2014, 90,000 students worldwide took the AP Physics B exam; of these, 60% passed, and 14% earned 5s.  (That itself is a bit down from previous years, because the number of students taking AP B doubled over the previous decade.)  That works out to about 13,000 students earning 5s on Physics B, and 55,000 passing.

In 2015, 170,000 students took the AP Physics 1 exam -- just about double the population who previously took AP B.  Part of the intent of the redesign was to increase the pool of students who could handle an AP physics course.  Physics B was intended as a second-year course, and was so broad that it did not encourage serious, deep understanding.  Physics 1 is in fact for first-time advanced physics students.  Many schools appropriately replaced their "honors physics" courses with AP Physics 1.  Good.

But this twofold expansion in the student -- and teacher -- pool means a much broader range of student -- and teacher -- ability.  Many of the 80,000 additional students taking the exam were intrinsically weaker students.  And a bunch of teachers who were not experienced with college-level physics, or who were simply not yet capable of teaching college-level physics, were nevertheless thrown into an AP 1 course.  No wonder only 4% of the country earned 5s; no wonder only 37% passed.

Let's look at raw numbers now, not percentages.  On AP Physics 1, about 7,000 students earned 5s.  This is about half as many as earned 5s on Physics B.

But 63,000 students passed the AP Physics 1 exam -- that's considerably more than the 55,000 who passed AP Physics B  the previous year.  Even on a more difficult exam, even with higher standards for passing, more students passed this year than last.  Of course... because Physics 1 is intended as a first year course.  Sure, a bunch of folks tried this exam who weren't ready (or whose teachers weren't ready).  So what.  Thousands of folks who were ready just fine tried the new exam, and found out that they could do it.

As teachers forcibly learn that physics is about more than plugging numbers into equations, as students figure out that they can't write a bunch of baloney and expect to earn credit, AP Physics 1 scores should eventually improve.  It's on us to adjust our teaching to help these scores improve.

44 comments:

  1. Wow! Those are some eye popping stats, but it really explains why more of my students didn't get the 5 I expected. The biggest issue for me was students taking the course that were not developmentally ready, plus guidance and admins were not fully aware of how the new course would be more challenging. Many students and parents see the word "Introductory" and underestimate the course, but I love the new exam and think this is way better than AP B for understanding physics.

    ReplyDelete
  2. I fully agree. I think the new structure is just what was needed. I also used that wonderful workbook as a main resource, relying on the collective wisdom of those who know more than I do (most everyone, I think). I took the approach of going straight through everything, rather quickly, and then going back over it in greater depth. I had 6 students who had taken honors physics already and I made them leaders of small study groups. Much of the testing, both FRQs and MC, was done in cooperative groups, which I think I would re-use in the future.

    My scores this year went three 5s, one 4, one 3, seven 2s, and six 1s. I went to what I think was a really good workshop up at St. Johnsbury and I hammered my kids all year long about how they were going to need to explain and justify everything. As you can see from my results, some took it to heart, but many either couldn't ... or wouldn't ... make the transition.

    I have a heap of ideas to make improvements next year, and if anyone wants to talk about the pop-up toys as an open-ended lab, I have used that one twice now and feel that I got some really good results from it. I have also converted some of the "useable" old FRQs to open-ended labs, which I find brings out the creative thinking in students.

    However, I did get an e-mail from my admin about how we will "talk in the Fall" about how I can do better, so that's kind of disconcerting to me. I have just finished going over the score reports and honestly the only real concern I have about my own effectiveness is that there were way too many 2s that "could've would've should've ..." and that's where I think I need to concentrate my efforts this year and find new ad effective ways to bring them along and up.

    ReplyDelete
    Replies
    1. Jeff - I'm interested in talking more about the open-ended labs you mentioned. Would email be the best way to do that?

      Delete
    2. Can someone share the name of this "wonderful workbook" that several have mentioned in their posts?

      Delete
  3. I am a first year AP Physics 1 teacher from Pekin, IL. I would love any additional resources to help my students be successful on the new exam. If you are willing to share those ideas, please let me know and I can forward you my school email address.

    ReplyDelete
  4. Happy to help... Just contact me by email.

    ReplyDelete
  5. Last year was my first year teaching AP Physics 1. I come from a VERY small school (about 50 students in a graduating class). I started with 6 students, but ended the year with only 1. This was nice since I could tailor the course to exactly what he needed, and not just him, myself as well. I STRUGGLE with rotational motion, so we both had to one another. It was probably the most fun I've had teaching in years. Long story short, he earned a 4 on the exam. After looking at the released FRQ's I was very impressed how well he did. This year I started with 9 students (down to 5 now). I've learned so much from the previous year, and my student from last year came in and explained the importance of self-teaching when concepts get difficult. He explained he did so well on the AP test, because whenever he didn't understand something I tried to teach, he would scour the Internet for various resources.

    ReplyDelete
  6. I think you are way too optimistic about exam scores improving. What I see is that administrators do not necessarily care too much about the scores because high school rankings are more skewed towards participation rates than exam scores, furthermore there is no benefit in most high school rankings for higher exam scores. As a result, administrators are packing these classes with unqualified students. Even if the teacher wanted to teach a more rigorous class, once students start dropping the course, the teachers will be reigned in by administrators. Teachers get the message and teach easier courses.

    By declaring that this course was a first year course, they opened the floodgates and put most teachers in an impossible situation. Last but not least this is just one of the many issues with this redesign. AP B had many flaws, but I also think it was a decent course with some really good questions. If the issue was too many easy points, why not just eliminate them or increase the cut scores. What happened instead was an ideological takeover rife with of conflicts of interest. In my opinion the net effect was the College Board damaging its brand.

    ReplyDelete
  7. I just went over the 2015 Physics 1 test that College Board posted and I feel that it is a very difficult test. Much more difficult than the Physics C Mechanics test. On a blog that I was perusing, the author stated that college professors who gave their students the test were pleased at the depth of knowledge that the test assessed. I agree, the test certainly does probe to great depths, but are those same professors only giving 4% of their students A's (5's on the test?). According the the College Board's own literature, a 5 on the test is supposed to be equivalent to an A in a college level course. I do not think that this is the case on this test. I had students earn 5's on the Physics C test that would have probably barely earned a 3 on the Physics 1 test.

    ReplyDelete
  8. Aaron, you're point is well taken... but the College Board has changed its philosophy as its redesigned its science courses. Previously a 5 was considered to be the equivalent of an A in a "typical" college class; now a 5 is considered to be equivalent to an A in the "best" college courses, or at least those that use "best practices."

    See my Jan. 23 2016 post for further details.

    ReplyDelete
  9. Of course the latest practice test for AP Physics 1 released by the college board reduced the score needed for 5 down to 64% which is about where it was for AP physics B. IMO the college board knows they messed up in 2015, and even though their tweets indicated they were doubling down on this test and how it was scored, I think they got the message from the high schools to either lighten up or get dumped.

    Why do you insist on defending the college board on the debacle that was the 2015 AP1 test? They are even tacitly admitting defeat on this one by ratcheting back the 5 score on the latest practice test. The 2015 test was flawed, biased against ESL international students, many questions were confusingly worded, and the ridiculous cutoff scores made it basically invalid from my point of view. I feel like all the kids who took it in 2015 got ripped off. I wonder how many millions of dollars were lost because students got deflated scores that didn't qualify them for college credit?

    Guess we'll never know. But I do believe one thing: next summer the college board will likely be crowing about how amazingly the scores improved in only one year! But it will only be because they lowered the cutoffs back to where they should have been to begin with, and they scared off all the marginal students who saw the artificially low pass rate from year 2015.

    ReplyDelete
  10. OK, anonymous, woah there. This comment section will NOT be used for personal attacks. Anyone else wants to post in this section, please put your name on your post.

    Neither teachers nor their students are being oppressed by AP Physics 1. The people involved in producing the exam are good people, doing what they think is right. You do not have to agree with them -- and I often don't. But I will not aid and abet those whose unsubstantiated conspiracy theories full of vitriol seem more concerned with sophmoric, "stick-it-to-the-man" rhetoric rather than reasoned discourse.

    If you don't like AP Physics 1, don't teach it. I and this blog can suggest many alternative approaches to introductory physics.

    If you aren't good enough yet to teach AP Physics 1 successfully, then I and this blog can help you get better.

    If you are just looking to rant, neither I nor this blog will be useful to you, so in that case please take your thoughts elsewhere.

    ReplyDelete
    Replies
    1. To begin, I did not post the "anonymous" comments above. At the same time, and personal attacks aside, I do think that he is expressing a frustration with the college board that many AP Physics 1 teachers felt after seeing their students' scores and the overall score distribution.

      When you consider that only 4% of exam takers earned 5's and only 37% passed - both percentages that are lowest across all AP Exams - I think it is only fair to question the efficacy of the AP Physics 1 exam.

      If "anonymous" was a bit too harsh in blaming the college board for these results it may be because the initial posting by Greg Jacobs failed to acknowledge any possibility that the test might have some flaws and that the cut scores might be a bit too high, instead pointing the finger at unprepared students and unqualified teachers.

      As a 15 year veteran AP Physics C teacher I would like to think that, if I gave a brand new assessment that only 37% of my students passed, I would want to rethink my both my instructional approach AND the validity of the assessment. Isn't it reasonable that the college board do the same?

      Delete
    2. Matt, these are fair points. My own take, having taught the course for two years and been on the periphery of the course development, is that the College Board's new approach to physics is so utterly revolutionary that it's going to take a few years for people -- teachers and students -- to properly change from a calculational to a conceptual approach to the subject. Yet I think that the AP Physics 1 approach is, in fact, superior to the old physics B style approach. Enough superior that I'm willing to live through a few years of adaptation.

      Should the College Board "re-evaluate the validity of the assessment?" Of course. But not yet. Right now, there are far too many teachers and students who didn't make any attempt to adjust to the transparent demands of the new course. In a few years, if the scores are still just as seemingly out of whack amongst classes whose teachers are making good-faith efforts toward meeting the expectations of AP Physics 1, then it will be time to make changes. But right now, for me, it's time to get better every year, and to trust the process.

      Delete
    3. Greg, thanks for the quick reply. You are absolutely correct that the new approach will require an adjustment period for everyone. As a teacher of AP Physics 1 I am still adjusting to the idea that my grading practices for this course are more in line with my colleagues in the English department than the Math department (I'm reading/asessing student writing more than I ever thought I would as a physics teacher).

      One of my specific concerns, and one I have experienced first hand, is that students are going to be discouraged from continuing on in physics. I had two students who earned "4's" on the 2015 exam but who were great students who basically told me that they would not be enrolling in AP Physics C or studying engineering in college because, "If I cannot get a 5 on the Introductory AP Physics Exam then physics is probably not for me".

      To clarify, these were both outstanding students who had only earned 5's on their other AP exams. Was their perception accurate? No. Then again, high school kids don't always see the big picture - despite my attempts to assuage their frustrations.

      Which brings me to my most important question. Who are we marketing AP Physics 1 to and are AP Physics 1 and AP Physics C standalone courses or should they constitute a progression. This topic was much discussed at the AP Physics 1 workshop I attended this past fall but we never really answered the question.

      At our school we (I) teach both AP Physics 1 and AP Physics C. Though we never taught AP Physics B (too broad, too thin) we marketed AP Physics 1 to college bound honors level students who may want to study science in college but not necessarily physics/engineering. Our hope is that some of these students will decide after taking AP Physics 1 that they really like physics and will go on to take AP Physics C as juniors/seniors concurrently with calculus. I'm just not sure that this is going to happen - but then again, maybe that is not the intent of the college board.

      I very much would like to know your view on course progression but please feel free to point me to another forum thread that might better help me process and think about this issue. On a related topic, it seems that colleges and universities have a pretty varied approach to their credit policies for AP Physics 1. I suppose that given the newness of AP Physics 1 this is somewhat to be expected but it does suggest to me that high school teachers are not the only ones struggling to make sense of just exactly what AP Physics 1 is supposed to accomplish. Thanks in advance for your thoughts and perspective.

      Delete
  11. Does anyone know what the estimated cut-off scores or % for 5, 4, 3, 2?
    Jim Bean Carson City, NV

    ReplyDelete
  12. In 2015, it was about 70% for a 5, 55% for a 4, and 40% for a 3. These are all about 5 percentage points higher than on AP Physics B exams.

    ReplyDelete
  13. Starting Circuits today ... 3 class periods. I have 4 90-min class periods to review. I will be using "500 AP Physics 1 Questions to know by test day" to review

    Jim

    ReplyDelete
  14. Jim, sounds great... know that I had nothing to do with "500 AP Physics 1 Questions to know by test day," and that the previous version (for physics B) was abominable. I do not in any way recommend the "500 AP Physics 1 Questions" resource. Now, they might have done a better job on this edition, but I don't know.

    There will likely be a "premium" edition of a future 5 Steps: AP Physics 1 that will include some interesting and unique practice problems sort of in the style of the exam. This is in the works, perhaps; I'll let everyone know if it happens.

    ReplyDelete
  15. This comment has been removed by a blog administrator.

    ReplyDelete
  16. Greg ... is AP Physics 1 kind of like Hewitt Conceptual Physics on "steroids?"

    Jim Bean

    ReplyDelete
  17. Yeah, that's a good way of putting it. I've used a lot of my conceptual physics materials as a basis for AP Physics 1 materials.

    ReplyDelete
  18. For next years AP Physics 1 students: their summer assignment is going to be to review all the
    chapters in Hewitt that we will be covering

    ReplyDelete
    Replies
    1. I use Hewitt a lot!! Only, I go into more depth in a lot of areas. AP Physics 1 is conceptual physics on steroids.

      Delete
  19. Greg,

    Thanks for the time you put into your blog. I really appreciate reading what you and other have to say. In the initial post you spoke of a letter on the "AP Community" blogs. Do you have a copy of this or a link? I tried searching for it but could not find it. Thanks and good luck to all you kids tomorrow!

    ReplyDelete
  20. I can't find a link. I believe it is under the AP course audit. Feel free to email me... and thank you for the kind words.

    GCJ

    ReplyDelete
  21. This comment has been removed by the author.

    ReplyDelete
  22. Results came in (July 2016) AP Physics 1... 51% received 3 or higher, average score 2.719
    5 = 3%
    4 = 15.6%
    3 = 34.3%
    2 = 43.7%
    1 = 3%

    ReplyDelete
    Replies
    1. I had 32 students take the exam

      Delete
    2. Thank you. Do ypu know the cut off for a 1,2,3,4,5?

      Delete
    3. Jim, where did you get that breakdown?

      I found the following breakdown as a preliminary indication (via Total Registration, based on a tweet from the College Board's head of AP, Trevor Packer: https://www.totalregistration.net/AP-Exam-Registration-Service/2016-AP-Exam-Score-Distributions.php ):

      AP Physics 1 2016 Score Distribution
      5 = 4.3%
      4 = 13.6%
      3 = 21.3%
      2 = 30.7%
      1 = 30.1%

      I searched for an official final tabulation but it looks like the College Board hasn't published their 2016 data yet. (I found 2015's data and the College Board website only has that available - nothing posted for 2016 yet.) Thus, the tweet above looks like the best indication so far.

      Delete
    4. College board ... I received my scores

      Delete
    5. Ah, gotcha! Your results, not overall/global. My apologies!

      Delete
    6. Alan,
      Globally: 2015
      5: 4.3
      4: 13.7
      3: 21.1
      2: 30.6
      1: 30.1

      Delete
  23. We are thinking about dropping AP Physics 1 after two years of offering the course. The disappointing scores that our students are earning combined with the fact that colleges are all over the board in how they offer credit is causing us to question the value of this class. I'm curious if anybody else is thinking about jumping ship?

    ReplyDelete
  24. I'm sure some are. Feel free to say so below.

    I'm staying with AP Physics 1. I found my scores were MUCH better the second year, once I figured out better what I'm doing. I'm still averaging 4s when I used to average 5s -- all the college physics people I've talked to seem well aware that the new exam requires a higher level of achievement to earn a 5. My best guess is to make the comparison, add 1 to the AP Physics 1 score to get an equivalent AP Physics B score.

    ReplyDelete
  25. Greg,
    Do you have the percentage scores needed for a 5, 4, 3 from 2016 for AP Physics 1? I want to place a score on their practice tests. I've been using the 2015 percentages you posted.
    Thank you.

    ReplyDelete
  26. J Groll,

    I don't believe that the percentage scores were published anywhere. I've been using the 70-55-40 cutoffs that are similar to the 2015 percentages.

    ReplyDelete
  27. Greg,
    What is your take, as an AP Physics reader, on the idea of circling or boxing your answer in Free-Response section? I've been telling my AP Physics 1 students to circle their final answers b/c I thought it would help the reader to know exactly what the student's answer is, rather than leaving it ambiguous and frustrating the reader to have to search for their answer. The issue I'm running into is that our AP Calc teacher is specifically telling his students NOT to circle/box their answers b/c the ambiguity might encourage the AP Calc reader to give more partial credit.
    Can you please clarify from your vast AP Physics reader experience?
    THX.

    ReplyDelete
  28. David, on one hand, your calc teacher is full of it. Clearer communication is always to be prized. Yeah, sure, there might be a student who is trying to BS a question who possibly gets a point of partial credit by fooling the reader. But for every such tricksy student there would be two or three for whom the readers - on their 1000th exam of the day - might miss the rightness of a student's work. Since my students, and I expect yours, are going to be right more often than wrong, go ahead and tell them to communicate appropriately. It's ridiculous, counterproductive, and unprofessional to teach students gaming methods to maybe once in a hundred times to get some undeserved credit for wrong answers rather than to teach them, say, good calculus. Oy.

    Now, that said... AP PHYSICS 1 DOES NOT HAVE FINAL ANSWERS TO BOX. No numerical calculations have yet appeared on the free response exam, and I'd be surprised if we see more than one or two per decade. So your question is essentially moot, anyway. Teach your students to communicate clearly and concisely, in fewer rather than more words, without BS. They'll do great. :-)

    ReplyDelete
  29. Hi Greg, any chance you still have access to the letter by Trinna Johnson and Trevor Packer? Would love to have a read on the score-setting process. Thanks.

    ReplyDelete
  30. Huh! I don't remember where I stored it. It was posted in summer 2015 to the "AP Physics Teacher Community", which is a College Board message board open to any AP teacher. If anyone can find, please post a link!

    ReplyDelete
  31. Hi Greg, I found the article you mentioned: https://apcommunity.collegeboard.org/group/apphysics/discussion-boards/-/message_boards/view_message/65299380

    ReplyDelete