Rude to observe? No, kids are used to it. Teachers are used to it. No biggie.
Rude to drop in unannounced? Not as long as it's a quick visit. All visitors disrupt the day.
Rude to observe unannounced? IMO, yes. And not indicative of true behavior.
If you want to get the low-down on why your child is struggling, call the teacher and meet with her. If you want to observe, the best way is to NOT let your child see you. I can tell you right now that with you being in the room, your child is not going to act like they normally do. I have seen the best behaved students go nuts with their parent in the room, and I've seen the worst behaved students clam up. It's not a good indicator of what truly goes on everyday.
But I do wish you luck!
I agree it's rude to show up unannounced. Our younger dd has mild autism, so I truly understand wanting to see her in the classroom (I'd show up everyday if I could!!) But I think you risk alienating the teacher early in the school year.
I would start off by treating the teacher like a professional. Show respect by giving her a heads up. I'd also strongly suggest trying to observe your child at things like lunch, where there's no academic pressure. See how your child is doing there too.
Their child was mortified, but if they had only been cooperative with the officer there would have been no problem.
I have 25 kids in each of my classes. I've had parents call me over stupid things like someone touching their child's pencil or breaking a toy that they shouldn't have had in school anyway. Yes, some of those things I miss. My focus is on teaching, not on keeping track of the contents of little Suzie's pencil box. 
