Well, when you say warm enough to swim, that's a loaded statement. I live in central west FL and frequent the world often. There are other posters on this board who live closer to Orlando and I hope they chime in.
If you are from the north (I assume you are) then late Oct early Nov is still warm to swim. March/April is peak time and I mean peak time. It's spring break and busy but the weather is usually just about perfect.
Other than that, the humidity is here. Late Nov, Dec and Jan are a gamble. We've had warm winters where northern folks would be ok swimming in. FL people usually don't swim as it's too cold for us.
Winters in FL range from warm (everyday in the 60's & 70's) to chilly in the 40's & 50's. But there humidity is non-existant.
It's still warm and humid here and it won't break until the end of Oct.
Hope that helps a little.