If you have a modern phone which has 3G capabilities then you can use it as a personal wifi hotspot (also called tethering). I'm sure this depends on the phone but I know it is possible with iPhone 4.
With my Orange contract I have to pay extra use the phone as a modem. For 200MB it costs an extra €8 a month. For 1GB it costs €19 a month.
3G dongles are OK as long as you have an HSDPA connection. This is slightly faster than 3G and slowly becoming more available across France. All of the various operators will have a coverage map on their websites.