I have gone in Late August with my son (now 10) in 01, 03, 04, 05, 06 and are booked for 07. We love that time of year as I do not have to take my son out of school to experience lighter crowds.
I will deal with heat and humidity any day over crowds. That being said, be smart and be prepared for that time of year. Take frequent breaks, drink LOTS of water, wear a hat, bring extra sun screen and remember it rains every afternoon (albeit briefly).
I thin most (not all) who experience difficulty that time of year just don't approach things right and try and do too much during the day. Our first trip, my son did get sick and we learned our lessons.
As for hurricanes, they are the exception in Central Florida, not the rule - I would not worry about them at all.