Do the arrows in free body diagrams have to be the right length?

No.

Draw the tail of the arrow starting on the object, pointing away from the object in the direction that the force acts.  Don't worry about the arrow's length.  Label the arrow with a descriptive variable, such as "T" for tension in a rope or "mg" or "Fg" for the gravitational force.  Off to the side, indicate the object providing and experiencing each force, as "T: force of the rope on the cart, and mg: force of the Earth on the cart."

What?  The length of each vector should be proportional to the magnitude of the force represented, as everyone knows, and as the AP exam occasionally requires.  And no one, let alone the AP exam, requires such precise descriptions of each force.  Whatchu talkin' bout, Jacobs?

I'm talkin' pedagogy.

Free body diagrams are one of the first physics skills I teach each year.  They're intended to give students a starting point for a three-step process:

   (1) Draw free body
   (2) Break angled forces into components, if necessary
   (3) Write two equations for newton's second law:
              (up - down) = ma_y
              (left - right) = ma_x

The physics teaching literature abounds with discussions of conquering a student's fear of mistakes.  In the educational culture that our students so often inherit from middle school, a wrong answer equates morally with being a bad boy/girl.  From such a source springs the trope that all physics teachers have observed: student proffers a blank paper, saying "teacher, help me, I don't know what to do."

The free body diagram provides a clear starting point to solve any force problem.  I can - and do - refuse to discuss a problem with a student who has not carried through all three steps in the procedure above.  Usually, once they attempt all three steps, my students suddenly find that they no longer have such an urgent need for "help."  And when they do ask for assistance, we can discuss the details of their problem solving process rather than the waily-waily woe of how impossible the problems are.

Thus, my requirements about free body diagrams must focus only on the elements that will most likely lead to problem solving success when students move on to steps (2) and (3) above.

I don't want students worrying about the length of their arrows.  That's not in any way important to breaking forces into components, or to writing Newton's second law in each direction.  In fact, it's not always clear at the very beginning of a problem which forces are bigger than others - often knowing the relative magnitude of forces requires significant reasoning involving an analysis of an object's motion, or consideration of the values of trigonometric functions.

Such reasoning can not be part of a student's initial approach to a problem - that's exactly how paralysis by analysis takes hold.  If I look at a student's solution and say "that's great, but the pushing force arrow seems longer than the weight arrow, and it turns out that the pushing force is smaller than the weight, so your free body is wrong..."

...then I've played straight into the waily-waily trope.  What the student hears isn't "almost right, missed one wee detail."  What the student hears is BAD BOY.  BAD GIRL.  YOU FAILED.

Similarly, I find it unproductive to think about starting the force vector at the point of application of the force.  (That is, a normal force arrow should start at the contact point between the object and the surface; the weight arrow should start at the object's center of gravity.)  This subtle convention is in no way important to solving Newton's laws problems in the first month of physics class.

I *do* want students thinking about the source of each force.  In order to be successful at steps (2) and (3), the free body has to be close to correct.  By insisting that students write the object applying and experiencing each force, I force* them to focus carefully on each arrow.  Without this focus mechanism, students tend to draw arrows almost at random, then hang their heads (BAD BOY) when they're wrong.  Don't believe me?  Watch your class work sometime.  I see, again and again, someone draw silly arrows (sometimes before even labeling them!).

* hah!

But if I let them struggle through without asking questions, I often see the same student stop as they start to write the source of each force, stare at the ceiling... then erase the incorrect arrow.

Since I started requiring a list of the source of each force in every free body diagram, I hardly ever see silliness like "force of momentum" in the direction of motion, or the "force of push" when an object was given an initial push, but is now moving without that push happening anymore.

But what happens when the AP exam comes?  It does sometimes require correct arrow lengths, correct starting points for arrows, and not any time-wasting list of sources for forces.

Gotcha.  The AP exam is in May.  I'm teaching free-body diagrams in August; I'm not wholly confident in my students' understanding of Newton's laws until we're reviewing in March or April.

At that point, it's simple to explain this added layer of complexity.  If required, draw bigger forces bigger.  If asked, put arrows for weight at the center of gravity, but arrows for contact forces at the point of contact.

As for the list of sources for forces... these are still worth writing, even on the AP exam, I think.  An ambiguous label like "F1" might not earn credit; but if F1 is clearly defined as "the force of person 1 on the ball", then that could hardly be clearer and will certainly earn credit.  More importantly, even on the AP exam, even after lots of prep, students tend to panic and revert to making novice mistakes.  By disciplining themselves to write the sources of forces - as they have been doing all year - they're more likely to find their groove, letting their knowledge flow.