We took each of our three kids before they were 1. DH and I are both Disney fans and enjoy going. It is a great place to travel with little ones, as Disney caters toward children of all ages. I have friends who aren't really Disney people, so they want to wait until their kids are old enough to appreciate it, since they will be going for the kids.
IMO, if you enjoy Disney and would like to go, definitely go with your 20 month old. You will have a lot of precious memories!