We took dd1 when she was 18months. It was a blast. We took our time, took lots of breaks, and realized that we weren't going to do all of the rides that we wanted, and may not be able to sit for nice long leisurely meals.
She may not remember it, but she loves to look at the pictures and talk about seeing Mickey.
We are going in Sept, and bringing dd1 who is now 3, and dd2 who is 9 months. It may be interesting with an infant, but this vacation is as much about us having fun as it is the kids having fun.
The other great thing about the world vs other vacation destinations is that there are plenty of things to keep little ones occupied, along with plenty of opportunity for R & R at the pool, resorts...
Have fun.