We've taken kids at 1, 2.5, 3.5, and 5. We always prioritized an afternoon nap and break from the parks at the hotel. Our kids never napped well in strollers, and the break is good for all. There's pros and cons about all ages, but each visit has been amazing and memorable. When we've gone with two, one parent stays with the younger child while another goes on a ride, and there's a lot to do all together of course. Some of my best memories are playing with my daughter while my wife went on a ride with my son.
I love WDW and will be there soon with a 6 and 2 year old. I will suggest, however, to think about
Disneyland with super young kids. It's so easy there with everything being so close together, lots of hotels walking distance, and the weather tends to be a bit better.