You are probably going to get a blast of various responses. Some people think anything under 5 is too young.
Here is my opinion.....
First evaluate if this is a trip you will make frequently, or once every few years, or less. If you are an infreqnet visitor, I would recommend 3 or 4 being the first trip.
BUT, if you go often then it all depends on the baby.
My DH and I have gone to Disney once a year since we met so for us, it was only natural to continue to go after having a baby. Our DD was 9 months old her first trip and let me tell you, it wa the BEST trip. She enjoyed it, although she probably would have enjoyed the mall just the same, LOL. But we enjoyed it, and we have PRICELESS pictures of her with all the characters. These pics are important now because even though she has been there 5 times now, only her first two trips (9m and 18m) have pics with characters because she is now afraid of them :-(
We aren't the go get em do everything plan it all out type. We just go with the flow and do what we feel like doing at the moment so if the baby was tired or hungry, we dealt with that and it didn't cause any stress of being off a schedule or anything. Our DD was very used to not being on a strict schedule and could sleep anywhere, carriage or strange bed, etc, so that is important.
So while I don't think there is any age that is the right or wrong answer, I do think it depends on your situation and your baby, etc....