Don't confuse decorum with fashion. I'm reasonably certain Representative Brooks was properly dressed when he beat Senator Sumner with a cane on the Senate Floor.
I'm also reminded of a story from some friends who were turned away from a three star Parisian restaurant for wearing jeans but graciously seated at a four star restaurant nearby. When commenting on this to the maître d', the proud reply was "We do not need to borrow class from our patrons." Humorous, but not without some truth. Attention paid to the other customers is attention not being paid to either your food or your companions.