Early November is a great time! June and August are still during the summer, so you'll see huge crowds. Early November is usually pretty calm, because all the locals are in school and Halloween time is over, and you'll get the first part of the Christmas decor.
Crowds in November are pretty low, the major exception being Thanksgiving week. So as long as you are there early in the month, it should be great! Shorter park hours, but much shorter wait times, so you can actually get more done.
As for weather, I think it's WAY too hot in the summer, so the fall is perfect. Should still be fairly dry and warm during the day, and you should be fine in a sweatshirt or light jacket and jeans at night.