One was 6, one was 6 months, and one was close to 1 1/2. Those infant trips were great. The younger ones took naps in their strollers so others got to take advantage of ride swaps to get extra rides in. Meanwhile they're a baby and there's more to distract them with in the parks so it was almost easier than just being at home. You push them around in the stroller, you don't have to carry things because it all fits under the stroller, and if they're in a bad mood they are both easier to distract and people don't get mad at you because it's a baby.
The ages that were the worst with all three of them were in that maybe 4-10 range. It's when they beg for stuff, get tired, sore, constantly change their mind - in the morning all they want is MK, and you get there and all they want is something at Hollywood Studios, you park hop and they just want to go to the pool. They stick to what they want when it's inconvenient, but before 4/5 years old they are just happily along for the ride. After around 9/10 they can be negotiated with.
Your 8 year old has a tantrum and every person within 100 feet is scowling at you, they can run off - or do the pout and refuse to move thing so you're dragging a kid who is acting like you're kidnapping them... I mean the good days are good. But the bad days with that middle age are just so much worse.