I just booked my trip for Sept 3 - 10 (Fri before Labor day to Fri after). We've gone this week for the past 2 years and have enjoyed ourselves. The parks weren't crowded - especially after Labor day. We didn't need Fast Passes and didn't have to wait long on longs and were even able to go on a few attractions 2 or 3 times in a row. Like a PP we found the first year that the Sun before Labor day was crowded in MK.
It is hot, but was bearable the past 2 years. We order cases of water from Garden Grocer - freeze them (we stay at DVC Villa) and they work great to keep us cool and thirst free in the parks.
It is considered hurricane season, but again we've lucked out the past 2 years - no rain or bad weather.
It has traditionally been free dining time so if you want to eat at any particular restaurants make sure to book them early. Right now you can book up to 180 days out. Though the parks aren't crowded the restaurants are due to the free dining.
Last years the Wishes dessert party was extended past August and we were able to book it and enjoy it. (I hope they have it again this year). The past 2 years they had
MNSSHP the Friday before and the Friday after Labor Day (that's why we changed our stay this year to Fri - Fri instead of Sun - Fri so we can hopefully get to go to MNSSHP the first night).
It's value season so the room rates are at their lowest (except for special offers, etc). This week works for us because I don't have to use an extra vacation day from work (already have Labor Day off) and right now DS is in Pre-K and his classes don't start until the week after Labor Day.
My son was almost 2, almost 3 and this year almost 4 when we go and the fact that the parks aren't crowded and we don't have to wait on long lines is a big help - especially in Fantasyland at MK. We got to go on all the popular rides (Dumbo, Peter Pan, Tea Party, etc) and some more than once.
All in all - from my experience, I recommend this time of year to go to WDW.