We went last Christmas and stayed at VWL while we had a great time it was crowded. We went for Dec 26 through January 2nd. We were so glad that we did stay onsite so, we could take advantage of early entry. For example, we did early entry at MK. We were able to ride Peter Pan's Flight, Winnie the Pooh (2 times), Dumbo, Snow White's Scary Adventures & Mad Tea Party. Then, we walked over to Frontierland/Liberty Square to ride the Haunted Mansion & Big Thunder Mountain Railroad (2 times). We did all this before 9 am. We were meeting our family who were staying off site. It took them forever to get into the park. Finally, they got in and we did Big Thunder Mountain Railroad again. Then, their kid wanted to go on Space Mountain so, against my wishes we walked all the way across the park. Stood in line at Space Mountain for over 45 minutes at that point. After that ride the park was crowded. It was too crowded to move so, we left. It was 11 am at that point. It is harder to travel in groups too. So, the smaller your group the easier it is to get things done. This was on December 27th.
We can only travel during school holidays. But if I had the opportunity to go during a non-holiday week over a holiday week, I'd definitely would pick a non-holiday week. I know that I cannot travel during non-peak times so, we usually, make the best of it. We don't go to the parks everyday. We also do other things in Orlando. Basically, we just try and spend quality time together as a family. I love going to the hotels and looking at decorations. My kids love swimming in December somthing that would never happen at home.
I made all my ADRs in advance. Lines for the C/S places were very long. Oh, I remember one day we drove to AK (we had a car). The line to get on the tram was unbelieveable. We had slept in that day and arrived with the masses. So, we left and went miniature golfing. We learned to be flexible especially after making the mistake of sleeping in. Although, we needed it.
Good luck with your decision.