After much experimentation, we have settled on a yearly excursion always heading down on the Saturday after US Thanksgiving. The crowds are OK - sometimes gets busy, but nothing unexpected. Weather is great - not too hot, not too cold. The Holiday decorations are up at that point too.
The least crwded we have ever found it (with the exception of week following Sept 11) was in January - however, this time doesn't work for everyone in our group. The worst for us was August - unbelievably crowded and unbearably humid and hot. Ugh.
After a lot of soul-seraching we did go the week following 9/11. At one point it was almost eerie it was so empty, and then on the next day which was a Saturday, they had to close the park as they had reached capacity.
I guess you never know - but still just after US Thanksgiving from now on.