The humidity! Well, that and the heat. We are not at all used to the humidity though, being from the Rockies, so it really wipes our kids out. The humidity sets in about Mid to late May and is there till about the end of October.
Then we look at crowd sizes/holidays. We travel with little kids and hour lines are just not in their plans. So, we avoid peak times and all summer, plus holidays. The park is just so much more enjoyable when there are fewer people. I am not saying empty, but just about a 4.
As for which days I go where. We avoid EMH days pretty much, unless we have to see Spectro. We find MK is best Tues, Weds and Thurs.
We also are big fans of getting to the park gates at opening. That seems to make the day in whatever park much better.
Hope that helps!