I'm not married, but I was engaged to someone who did this. That among many other things ended our relationship. The thing is - I was heartbroken and utterly destroyed, but my reaction wasn't extreme anger, like "don't let the door hit you on the way out." I still loved this person, deeply, and losing him was like losing a part of me. I guess what I mean is that even when someone has been so cruel to you, you still love that person, and it's hard to cut them out of your life automatically. It was more hurtful for me than for him.
It's kind of like children who are mistreated by their parents - many of those children still love their parents and don't want to be taken away from them, even though they are treated badly. Not saying that it's okay or that it shouldn't be a deal-breaker, just that it's often more complicated than just "get out."
I've been on the other side of this too - I was with a man for quite a while who I didn't know was married. My reaction was more of anger, but it was mostly sadness. It's always tragic to lose someone you love, and especially so in these circumstances...