I also talked to my children this way from the time they were young. If they asked me a question, I said, "yes, ma'am" or "no ma'am". They are polite and respectful to everyone because this is the way they learned to talk. I can't tell you how many compliments I get on my children's manners. Be mannerly to them, and they will learn respect for other people. I don't think of manners as a way to "show off" or be superior...or even being from the south...it's just a way of letting other people know that you think they're significant. What's wrong with that?
I grew up thinking my elders were significant and worthy of respect and I've never called anybody ma'am or sir in my life. As far as I know, my PARENTS never called anybody ma'am or sir. (Well, my father was in the Air Force, so I guess he did for a while!)
So if you equate not using those terms with a lack of manners/lack of respect, you'd be mistaken. It's simply a custom in some places and not a custom in others.
To go even further... I don't think I ever heard those words out of my grandmother's mouth!