I took DD#1 when she was 3.5, then we went again when DD#1 turned 5 and DD#2 was 3.5. I am now planning another trip for this fall when DD#2 turns 5 and DD#1 will be almost 7.
These are some things I learned:
On the first trip, we took an early (7am) flight out of Seattle. We stayed at an airport hotel the night before, but our
travel agent advised us to be at the airport at 4am. I have no idea why. I felt like DD never really recovered from the energy loss of this early morning. The hope was that she would sleep on the plane, and she might have for about 30 minutes. From now on, I will always make sure that young children can get a full night's sleep before departure.
Our first trip, we stayed at a Harbor Blvd hotel what was about 10 minutes from the park entrance. It was cheap and fit the budget at the time. No real complaints. But the long, non-magical walk made me hesitate to go back to the hotel for any mid-day breaks. On our second trip, we stayed at the Disneyland Hotel. It was a splurge, as it was our first real "family" vacation. Nothing beats the magic of the DLH at Christmas Time. Your 4-year-old would LOVE it, even with the construction that is going on now. The walk through Downtown Disney is much more pleasant, and the monorail is a nice option. I probably wouldn't pay to stay at the Grand Californian with a 4-year-old.
First trip, stayed 4 nights. As I mentioned earlier, it started with an early flight so we were in the park by noon that same day. We also visited the park a bit on the last day before we flew home. 5-day park hoppers. On our second trip, we stayed 5 nights. We arrived around 5pm the first day. Didn't leave until noon on the last day, but by that time the girls were pretty wiped out, so we didn't even go to the park. We had 6-day hoppers but only used 5 days worth.
Honestly, the though of taking small children to Disneyworld overwhelms me. I know there are people that do it. But as a PP mentioned, you need a longer vacation to really enjoy what Disneyworld has to offer. I just think my kids would be exhausted by the time we had a chance to see what we wanted, let alone to revisit favorites.
So, in my experience, 5 nights seems like a good amount of time. It gives 4 full days and perhaps some additional park time on the first or last day. If the Disneyland Hotel is in your budget, I would totally go for it. If not, I know a lot of people here like HoJo, but I just can't do the walk again with a young child. (I didn't stay at HoJo, but at a hotel an equal distance in the other direction.) For our next trip, I will go with either Disneyland Hotel or with one of the cheaper motels directly across the entrance on Harbor Blvd.
Have fun planning, that is almost as much fun as the actual trip!