In my opinion, the absolute lowest crowd levels are the two middle weeks of September and the first 2-3 weeks in January.
That being said, there's a slight risk for hurricanes in September (hopefully not like we had this past year), and it's a little chilly in January. The parks also close a bit earlier during these times.
Another great time to visit with only slightly larger crowds is right now. The first two weeks of December are very, very nice. All of The World is in full holiday swing and it's really something to see. More people around, but the parks are open a bit later and the lines are 15-20 minutes at most on average.