I've been at that time of year a couple of times. The weather can vary wildly, so if swimming is a MUST for your family, prepare them to possibly be dissapointed.
The crowds are usually very low, so you can accomplish a lot.
We went 12/16 and 12/17 of 2004, and we managed to fit in almost everything in all 4 parks. The weather ranged from lows of about 50 in the early morning, to highs in the low 70s mid afternoon. My kids did swim our first night there, but we bundled them into warm baths as soon as they were out of the pool.
One thing to consider is that there are often shorter park hours because it's not very busy. The Christmas party also shortens MK hours on many days.
Scope out the park hours on the official site, and prepare to be flexible. There are many different options at this time of year, with lots of holiday happenings (Storytellers in EPCOT, Candelight Processional, Osborne Lights in MGM, etc.) Enjoy!