I agree, is it a one time trip, or the beginning of many trips? But also, it depends on the temperment, dispostion and personality of your child.
We swore we would never take our DS until he was old enough to "walk around all day by himself", etc... but at age 16mos, he developed a fascination with Toy Story, and soon after, all Disney animated films. At age 18mos, we decided to bite the bullet and take him. We just went for 3 days (we only live 5 hours away by car), relaxed the first day, had breakfast at Chef Mickey's (which was a wonderful warm up to the characters, by the way), went to MK and Epcot on day 2, and did Downtown Disney before driving home on day 3. He truly enjoyed himself.
In April, we did an impromptu trip (literally "planned" it 45 minutes before we left), and at 26 months, he loved it even more. The next weekend, we went back (did decide the night before, that time!), and it was the best trip yet. He remembered enough to not be nervous about the rides and the characters. Now, he talks about our trips constantly. Remember this? Remember that? We are going in September, and he can't wait! (We are also going in December, but he doesn't know that yet.)
DH and I haven't ridden Rock n Roller Coaster in 3 1/2 years, and I don't even care! Will I be glad when DS is old enough to ride it? Sure! But right now, our trips are about what he wants to do and see, and we have discovered that doing WDW that way is even more fun that being a commando thrill rider.
However~DS is very laid back and relaxed, and is accustomed to us dragging him around everywhere. He is able to go with the flow. He seldom has meltdowns and tantrums. I know many toddlers that I could not stand to be in a car with for 5 hours, not to mention carting them around the World all day. DH and I neither one deal well with tantrums, possibly because we've never had to!
All of these are things to consider. I am so glad we decided to not wait until he was older to take him... but we know we'll be there a couple times a year, so we will get to experience it through his eyes at every stage!