I am Japanese and my father was very strict, especially manner wise, meaning I am also from old school
So I am oppose to wear hat anywhere inside building.
I personally do not want to see somebody wear hat in restaurant(even fast food). I also do not want to see some else's flat hair.
I do not make any action, but I assume those people as rude or disrespctful to others.
(If I wear cap/hat inside building my late father would come back from heaven to kick me

)
My understanding about eating outside or going outside is to deal with other people. I have to realize I AM NOT AT MY HOME.
So, any kind of my act may be offensive to others.
And manner is manner. I personally take meaning of manner as "social rules". I do not want to create my own rules in this society because my hair looks flat or ugly or whatever.
I also get flat hair after wearing cap/hat. So unless I really NEED it, I won't wear cap. If I have to, I bring my comp, dep, every single item to fix my hair because I CHOSE TO WEAR CAP/HAT