Filling out the tax forms is well worth the effort. Most countries tax their residents based on their world-wide income, so once you establish your "tax residence" in Australia, you'll be expected to report all income (including that on your UK bank accounts) to the tax authorities and pay tax on it.

There are a few types of income that continue to be taxed by the country where they come from (as Kaz has mentioned), but these are usually handled by treaties between the countries. If all you'll have in the UK is some bank interest, by all means, fill out the forms and save yourself the hassle of having to figure out the treaties once you get to Oz.
Cheers,
Bev