I agree with the PP that there is no right answer to your question.
We took our 4yoa DD when she was 13 months on her first trip to WDW. She went for her 2nd trip when she was 3. Both times were an experience in themselves. In October we are taking her again for the 3rd time along with her 13 month old ds (this is his 1st trip).
You know your kids best and what they like, what they don't like, and how much they can take. Only you can answer the question what is the best age to take them. Plus going to WDW isn't just for little kids it's for us big kids too! I don't care that Caroline doesn't remember. I remember the trip and the whole experience and we video taped it for her to watch and believe me she watches "baby Caroline goes to WDW" all of the time.
Caroline is so excited she can't stand it. Actually, I think that this trip was the reason that she is fully potty-trained. We told her that she couldn't go unless she was out of diapers and it worked.
Now ds has no idea what WDW is, and that's fine, but we want to give him the experience of new sights, sounds, and smells. He is very inquisitive and I think that he will have a blast.
I would however give you this recommendation.....if you go, go for a full week. On our last trip dd was in a funk for a few days and it totally interfered with our plans and we really didn't get to do much. I think with little kids you really need a full week so that you can take your time - you can go at their speed.