OP here... I just got back from speaking w/ father for an hour.
He was very nice. Invited me to come in (the parish offices) for coffee to sit and talk about it. He said he doesn't recall saying it (so at first he denied it, then when I said I heard him myself, he looked surprised but said if he said it, he certainly didn't mean it that way).
But he was trying to put it on my ds, saying "if your ds is so sensitive that this one word is upsetting him so much, we need to strengthen your ds. I'm concerned about him". I didn't like this at all and told him so (very nicely). I told him I didn't like that he was putting this on my ds and not taking responsbility for what came out of his own mouth (that's my exact words, but I said it nicely). I explained how today we parents, because of political correctness, have to teach our children to not say the word "stupid", and that if a teacher had said that to my ds, it would be a problem in the school. He said he truly didn't realize that. I told him it's the same as saying (please forgive me for saying this) retard (i actually whispered the word, and he laughed). How when I was a kid, if you were mad at someone, that's what you called them - we heard and said it all the time, and how today, you simply don't say it and we teach our kids to not say it. He understood more then. I also told him I didn't like that he closed the door, and he shouldn't have and I wouldn't have if I were him (not in a threatening way, just telling him how all adults need to be hyper-aware of being alone w/ children, and simply not do it). He agreed. We talked about the "safe environment program" w/ the kids, and that's why I overheard some things (because of course he said I shouldn't have been listening), but I explained I was standing there because he closed the door. He said it was an ackward situation because it wasn't a real confessional room he was using, and for privacy from the hallway and room right next door, he closed the door, but he even felt weird about it.
He wants to apologize to ds, and I said I'll only allow ds to speak to him w/ me there, and he fully agreed. He said 'let's have your dh come too', but I explained how we own a local business and dh is a busy guy, then it came out he's a long time customer of ours (my dh had said this to me, that he thought he recognized father, but he was far away w/out his glasses). Anyway, my dh had said to me "if it's the customer i'm thinking of, he's the nicest guy I know and I'm just shocked".
I told father how surprised I was because I loved hearing him speak "to" the kids instead of over them, etc.
He said a lot of things like "i'm so glad you came to me, let's please take this as a learning experience... what is jesus wanting us all to learn from this..." stuff that I'm always on the fence about w/ religion anyway. So part of what he said made me feel sort of OK (but not fully wonderful), and part of what he said was the religion part that I have a hard time with.
In the end, he didn't take responsibility for his words, because he kept saying he didn't recall it, and he didn't think the word was a big deal. He seemed more understanding at the end than at the beginning about it though.
I've been researching homeschooling in my area for the past few days (always getting ahead of myself), because that's really my only option. I wouldn't do our public school, and the other private schools around here are all religion-based, which I wouldn't do again if we left this school.
I like everything else about this school (well, most everything - every school has it good and bad points), but we were mainly happy here before this. BUT i've always thought about homeschooling in the back of my mind, and this has made me really start thinking about it again. So of course dh says "people think we're weird enough already for goodness sakes!" Yea, he's right on board w/ me!

IT'S SO HARD deciding things for your kids!!!!!
I was ready to pull the kids out today if I had to, if father was like 'too bad, so sad!' (can you imagine him really saying that?!?

), and homeschool them the rest of the year to see how we all liked it, with the intention of... I don't know what... in the fall. sigh....
so I guess we're ok... but I'm not 100% happy, but will I ever be anyway?