Yes, sometimes mine comes off as well. I have a particular student who gets aggressive. When this happens, he has been known to pull on my lanyard which hurts my neck a lot. I will remove my ID when he grabs it, and have on occasion forgotten to put it back on. It is not out of disregard for the rules, but in and of itself is a safety issue. (If I can see it coming, I will simply tuck it in to my shirt.)
ETA: I have a tough room this year. I have already been bitten twice so hard it broke skin (and have been bitten several other times.) I have had things thrown at my head, been kicked, had my face clawed, been spit in the face, hit, etc. When this is going on, my ID badge is the last thing I will worry about. My priorities are the safety of the other students in the room, the student who is having problems, and myself.