Basically, no, you do not need a long-stay visa to join your British spouse in France. You're covered by different rules than someone who is married to a French national.
You will be joining your spouse, who is "exercising his EU rights to live and work (or study, or whatever s/he is doing here)." (It's a strange phrase, but it's important.)
For that, you need to make very, very sure that your US passport gets stamped on your entry to France. It's the standard "Schengen visa" that you would get if you entered France from the US - and you need it only to show that you have legally entered France and when.
You then have two months to go to the prefecture and request a carte de séjour as the spouse of an EU national currently residing in France. This is the relevant page from the Service Public site:
Citoyen européen ou suisse : installer sa famille proche en France - Service-public.fr
You'll need a bunch of documents (including your birth certificate) for yourself, and also certain documents for your spouse, like passport, proof of whatever s/he is doing in France, proof of residence. Safest thing is to check your local prefecture's website to see if they give you any guidance, or go to the prefecture shortly after your arrival and ask them for a list of what documents you will need to bring (when they'll also tell you what needs to be translated, etc.). Or, ask your spouse to ask at the prefecture for the list of what documents you'll need to apply for the carte de séjour.
You shouldn't need any documents from the embassy if you are shipping your belongings from the UK. (That's the joy of the EU!) If you're shipping stuff from the US, you may want to arrange to have your stuff arrive AFTER you have received your carte de séjour, as it will make things a bit easier. (Not sure if you'll be entitled to quite the same consideration as someone moving to France on a visa, but if you ship stuff using a mover or shipper who is handling the customs clearance for you, they should be able to advise you.)
Cheers,
Bev
