The two main things to get from the seller are the carte grise (now called something else - but it's the vehicle registration document) and the paperwork from a recent controle technique (must have taken place within the past 6 months - but usually is done by the seller once the sale has been agreed upon).
Note, too, that you need 3 originals of a Cerfa form that documents the sale of the vehicle - one for the seller, one for the buyer and one for the prefecture.
For your information the Carte Grise is now called the "Certificat d'immatriculation" - much less imaginative. Although I was asked to show mine to a Gendarme the other day and he asked to see my Carte Grise.
They changed the name at the same time as the number plate system. Before when you bought a car with an old number plate (e.g. 8797 XZ 74) the number plate changed with each new owner. Now the number plate stays the same for lifetime of the vehicle (e.g. BB 767 CV).
You can also choose which department number you would like on the number plate.
Actually you can now do it through your mairie and the easiest is that you ask them advise there if you're not in a big city.
You need to give them the old carte grise (or certificat d'immatriculation as the admin people will call it) cross with the note "VENDU LE (DATE) and "TIME) and you also need to provide the copy of the CERFA that say "Exemplaire prefecture" and a valid less than 6 months old controle technique.
You should also ask the seller for a certificat de non gage(meaning he doesn't own any money on his car for a credit or something else). Ask Bev said there are 3 copy of the Cerfa (one for the seller to keep, one for you and one for you to hand out with you paperwork to the prefecture).