How would I feel????
I would not want to be accepting of any of this, at all.
What you have described are not the behaviors, words, actions, or attitudes, of a committed or trustworthy person.
I wouldn't have to have cold, hard, proof, after the fact, of any really truly clear infidelity.
The thing is.... At my age, I also know that there are a LOT of woman who are in a position where they simply cannot (or feel very very strongly that they cannot) leave a marriage. There might not be anywhere for them to go. They 'learn to accept' a LOT.
Whether you choose to stay, for the sake of staying... Or.... yes, you could continue on, as you have been, risking him finally, tomorrow, or next week, or next year, deciding to divorce you and change the locks... That is a very personal decision. And, I would not begin to give you any specific advice.
My words of advice???? I know that hope springs eternal. But, at this point... where you are at, after 20 years... I would caution you very VERY strongly against 'hoping' and continued commitment and emotional commitment, that is based just on a possibility that he makes up his mind, finally, that he does love you, that he is able to 'be married', and that he is committed.
PS: I do not go with 'putting on notice', or demands/ultimatums, either.... One can not control another persons attitudes, and even actions. Only your own. OP, you need to decide what attitudes and actions YOU are going to take. You need to look it from the angle of, what can you accept, what do you want to do... what do you need to do to be at peace and happy in your own life.... Not, what are you going to make/force him to do.