My DH used one on our trip. Some people probably thought he looked like a 51-year old dork, but he didn't care! He wears glasses and needs a place to keep them safe during some rides. He also has prescription sunglasses and frequently switches between those and his regular glasses. I could carry his glasses in my bag, but he would be constantly asking for the case so he could switch his glasses (annoying for both of us!). A fanny pack is a perfect solution. It's small, light, and keeps his hands free. His glasses, wallet and a few other little things fit nicely. He has no desire to carry a shoulder bag of any type, and he doesn't have enough secure pockets in every pair of shorts he owns.
If a fanny pack is convenient for you, go ahead and carry one. Anyone who thinks it's silly is probably secretly wishing they could rock one too!