I wanted to take mr DGD when she was 3 just as my DH and I had taken her father as well as our daughter (the 3 yo's aunt). But DS and DIL thoght she was too young and then they had DGD #2. They made us a promise tho that they would not take them or let them go with anyone else and that we would be included when the girls finally were old enough to go but they wanted to bre there when the girls saw the castle for the first time. DILs brother lives about 1 hr from WDW.
I thought the day would never come as DS isn't fond of Disney at all. However, next week, DS has a conference in Anaheim, only two blocks freom DL and he has invited us to go with him and plan a DL trip for all of us.
Then, to make things even better, he has agreed to go to WDW this Christmas when our entire family (about 32 of us) gather at BLT for a grand gathering. I will finally get to be at WDW with my girls!
To give a short answer to the question: 3 is about as young as I would take our girls to WDW and thankfully, my DS and his wife don't have a problem with allowing his daughters to go on vacations with me and my DH or to come here where we live (8 hrs from them) and spend a week at a time. DILs parents live 20 minutes away from them.