I am sorry. I think your original response to your parents was great. BUT, your parents are abusive, and THEY responded just as your pastor suspected they would.
Personally, I think you need to throw your hands up at this point. And realize that threatening to take care of them when they become old and ill is not going to help things!
In the first place, not everybody needs family to step in to help them when they become old, and in the second place it's essentially none of your business if your parents don't choose to prepare for retirement, old age, or illness. I would absolutely flip my gourd if any of my blood-related family tried to step in to "help" me in my old age--how DH and I prepare for our old age is ONLY the business of the people we are closest to (ourselves and a few very close friends who are much closer than blood family).
I also have to question why you would WANT your parents to move closer to you so you can "take care" of them. They are hateful to you. How would this affect your life and the life of your DH and children? They should not be subject to your parents hatefulness, nor subject to your anguish over the situation. They deserve your attention and care--your parents neither want it NOR deserve it.
If you feel you must take care of your parents in order to honor them, there are other ways to help with care from afar, like hiring a geriatric case manager if your parents become unable to care for themselves and important choices need to be made. You DO NOT need to be the caregiver yourself. And they have released you from any obligation by informing you that they do not desire your help.
Your parents clearly do not want to have a relationship with you and your family and you can't force it. I'm puzzled and saddened that you are still trying. Don't allow them to perpetuate their abuse of you. And don't be an example to your children of how people can be abused over and over again and come back for more.
Most of all,

to you, and I am sorry you're going through all this.