This is from the tax information booklet for 2008 put out by the Paris IRS office:
>>If you have not filed a U.S. income tax return for one or more years
and no money is due, you should file returns for the current year and
two prior years (i.e., 2008-2006).
However, if you have not filed a U.S. income tax return for one or
more years and money is due, you should file returns for the current
year and five prior years (i.e., 2008-2003). Instructions for getting prior
year forms can be found on page three of this booklet.<<
The three year back filing thing is a long-established concession extended to overseas taxpayers (certainly for at least the last 15 or 20 years).
It's based on the concept that if you "voluntarily" file, they are more than willing to forgive and forget. If they find you for failure to file (generally because you owe a significant amount), then all penalties apply because there is no statute of limitations on returns not filed. (And that's how the IRS officials put it at the tax meetings I've been to here in France.)