You have picked THE perfect time to go!!! You miss the "Jersey week" crowds, Super Soap Weekend crowds, and crowds from the F&W festival.
We have gone the same week many times. You will get to also enjoy less crowds because you will be gone before the Thanksgiving crowds arrive.
The lines will be 5-10 min or less. It is not uncommon for us to just walk on some rides. The more popular rides do have lines but not all the time and fast pass usually helps with that. The weather usually is between high 70's low 80's. We always swim in Nov. The afternoons definately get hot enough and the pools are heated! I usually wear capris and t's and DH and the kids always wear shorts and t's. At night it does get alittle cool sometimes. So, we bring a light jacket and I usually have 1 pair of pants for each of us just in case.
You are going to love this time of year and be spoiled by the lack of crowds and beautiful weather! Oh, and you'll get to see the resorts decorated for X-Mas and the parks transform around that time too!
We are in the planning stages for a trip that week too!!