We went this past November 11-17. I was disappointed by the crowds. A couple of our days were very pleasant crowd-wise (Epcot one day, MK one day), but the others were much more crowded than they were two years before. I still loved it and would go back again, mostly because November and December are amazing, but don't expect many or any walk-ons. We had two level 3-4 days, and one level 7 day, which was a zoo.
That said, we have lucked out with WDW weather at that time of year. We had a string of days that were between 72 and 84, sunny, perfect for an afternoon swim, and nice and cool in the evenings and mornings. Only one day of rain, and it did pour that day, but generally, that's the dry season, and not humid at all. I can't be in heat above 88 degrees without feeling faint, so I would say November or early December is perfect weather-wise. Plus, if you go after November 7th (or so), you can see the Osborne Light show, which is amazing and truly magical to behold.
If I had to do it again, I would go immediately following Thanksgiving, but it would mean slimmer chances of swimming, which our boys love.