March and April bring spring breakers and other larger crowds. I'm not sure when Easter is this year - but that is a factor too.
We have gone in late October. It is wonderful. The weather is warm and the crowds are quite small - except maybe right around Halloween - but usually then only at the Magic Kingdom.