We went the first week of November last year. I found out about the infamous JERSEY WEEK and worried, since we were planning around a friend's wedding and couldn't change dates. It turned out fine. With planning, and getting to the parks early, we did and saw just about everything we wanted. Got to do most of all four parks, TL, DTD, and some swimming at the resort, and we were only there four nights/ three days! Weather was nice (hotter than I thought it would be), crowds weren't too bad. I'd go again that time of year- we'll be there again this Oct 29- Nov 4!
If you are going between the beginning of November and about the 6th, you should be in good shape. This year, the second week will be the crowded one. Jersey week, Veterens Day, Super Soap weekend, and the tail end of Epcot's Food and Wine Festival all fall on or around the weekend of Nov. 11- 13. Expect the parks to really fill up that week and weekend.
Oh! Editing to add that this year, Nov. is considered Regular Season- in years past it was considered Value Season. So resort prices are higher this year. Discount codes aren't out yet, but I'm guessing there won't be so many good deals this year. Changing to Regular Season seems to be an indicator that occupancy has increased to the point that Disney doesn't have to offer as many discounts to get people to the parks... they are already coming!