I can confirm that that week is the infamous "Jersey Week". The schools are closed 11/9-10 (Thursday-Friday) due to the teacher's convention, and some schools, including ours, has 1/2 days 11/6-8 (Monday thru Wednesday), therefore most folks end up taking vacation that week, especially to Disney. They either leave the weekend before (which is what we usually do) or they leave on Wed. eve. Therefore the latter half of that week is the most crowded.
Typically, though, it was never REALLY crowded then, since most other folks DIDN'T go then.
I will say though, that I found that week to be a little more crowded last year than usual, and I had asked a CM if they felt it was more crowded than usual. He said that he had noticed all of November to be more crowded than usual, and that he thought it was due to people taking their vacations later in the fall, to avoid the hurricane season.
It is a BEAUTIFUL time to go and I think that if you are going earlier in the week, you should be ok.
Have a great time!