If you are physically located in France while doing the work, then you are "working in France." Where the employer is located has nothing to do with it.
To telecommute like what you are proposing means that you would be working as a contractor (or, if you prefer "consultant") and not as an employee of the US company. Basically, you bill the US company for your services, and they pay you what you bill, with no taxes, withholdings or benefits taken out. You then are "self-employed" (and in France that means you have to set up some form of business entity to handle the taxes and cotisations) and responsible for making your various payments.
If you expect to take in less than 32,600€ in the first year (yeah, you have to convert from $), then you can start out as an auto-entrepreneur. Details here: Portail Officiel des Auto-Entrepreneurs
If you will be paid more than the limit, then you have to look at establishing something like and EIRL or EURL and that means you'll have to register for VAT and add 19.6% VAT to the invoices you send to your employer in the US.
The AE is actually a pretty nice and easy way to go if you qualify. You get into the French cotisation system and your cotisations and taxes are simplified. On the US side, you declare your gross income and then take the Foreign Earned Income Exclusion, so shouldn't wind up with any tax obligation to the US - though you never lose the filing obligation. (Take a look at the Expat Tax section here for the details on that side of things.)