Just look at your car's paint in the noon sun. You'll be able to see any swirls, most of which will be put in by washing and wiping with a towel. A microfibre towel can also scratch, especially if it has dirt stuck in it.

A way we use to test how gentle a towel is, is to wipe it against the reflective side of a brand new CD-R. Good microfibre towels won't scratch.