if I was only going to be able to afford to go once, I would wait til my youngest was 7 or 8 and could go on all the rides, have an easier time with all the walking and be old enough to really remember the trip. It's hard to wait when you really want to go, though, I get that.
and if my kids were below say 7th or 8th grade, I would go in May when it's cheaper and less crowded. I was surprised that in the younger years teachers understood about taking the kids out of school for vacation. They gave us all the work.