I'm lucky because I learnt French at school and can speak it well enough to communicate with the French. That's one reason I have a house in France. I don't speak Italian, Greek, Chinese or Serbian though, and I would never, ever dream of relocating to Italy, Greece, China or Serbia, or any other country where they don't speak English. How can anyone possibly expect to live in a country if they can't understand and communicate at a basic level at least? That said, I have the impression that most Bretons (I'm in Brittany) are very patient with the Brits who don't speak English. However, amongst my friends over here who don't speak French there are several issues:
1. They often have serious misunderstandings of what is being said/meant by the French....
2. They often feel really alienated and isolated as they can't communicate...
3. They are often frustrated when shopping, trying to get jobs, fill in forms and comply with regulations etc. as they can't understand or be understood...
4. Many (most?) jobs aren't open to them for obvious reasons...
5. Unscrupulous people can take advantage of them and they don't even know when it is happening - and misunderstandings can lead to what seems to be deliberate taking advantage....

and so on.

Whether or not it is easy to learn, it is necessary to learn at least enough not to appear rude. If you are fairly young, have lots of time and motivation and a good teacher or course, you can learn quite quickly to communicate well - say 2 - 3 years. If you are middle aged and have less time it might take a decade or simply never happen on anything except a basic "ca va? ca va bien, merci" level.