We had a trip planned for early Dec., but had to postpone due to moving. I didn't want to wait an entire year, but low crowds are pretty important to us (two of our children are autistic.) So I checked out the Unofficial Guide anticipated crowd levels, as well as last year's archives here at DIS.
I then picked the week after President's week. The weather should be pretty decent (we're from Maine, so anything above 50 seems downright tropical) and the crowds, while not September near empty, will be at their anticipated lowest at that time.
We also have 4 (ours are 9 and under, space every two years or so.) Good luck and have fun.