France only recognizes the civil marriage, which means you need to have the marriage license/certificate. In order for you to get a visa to join your husband in France, you will need to get a certified marriage license - which is issued by the State rather than by the local (town or county) authorities. This is required for obtaining your livret de famille (the French document that "proves" that you're married).
You should probably start checking now about how to get that certified marriage license. In some cases, you have to apply to the State (after the wedding, obviously), in others you have your local wedding license "apostilled" - which involves having the State authenticate and validate it.
You then submit that to the French consulate for them to issue the livret de famille and then you can apply for your spouse visa.