I was always taught that men shouldn't wear hats in a restaurant, and that is what we are teaching our son.
But honestly, it's one of those things that I don't understand. Someone decided years and years ago that it was rude, so now people have to abide by what someone else decided was rude a long time ago. Just doesn't make sense to me. I really can't think of one good reason why a person wearing a hat indoors is rude, other than that it has always been considered rude.
Not touching other people, not cutting in line, keeping your conversation down, giving up your seat to an elderly person, those are all things that if you don't them make sense to be considered rude, because they directly affect other people. Hat in a restaurant = rude, just doesn't make sense to me.
But like I said, I've taught my son that he is to always remove his hat while eating (unless outside in the sun). I just hope he never asks me why, because I don't have a clue what I will tell him. I guess it will just fall under, "because mommy said so".