I postponed our Disney trip because I thought it wasn't fair to our daughter. We only went when she was three (almost four) because my mother said she would be taking the kids, then backed out, and we felt that we couldn't reasonably disappoint the kids when they'd been looking forward to it all year. We all loved it so much that we've fit in as many more
Disney vacations as we could since then.
One thing I realized was that while I was trying to be fair to my daughter, I was unfair to my son, who is 4 years older. By the time we went to Disney, he was eight years old, and while it was fun for him, he was already starting to lose some of the fascination with the characters. I really regret not letting him go to the parks when it would have been more magical for him! My daughter has already forgotten most of the details of that first trip, but I was there and I know she had a great time. I think it would have been nicer if we had gone at least a little bit earlier.
There's lots and lots of things that your 20 month old could do. So many of the rides are family-friendly! Make sure to plan for down-time for both kids, and everyone will have a great time.