We were there the first week in May last year. It was hot but not humid and you could swim every day. The crowds were moderate and very manageable. We're going same time this year as we think it's a great time to go.
I was in WDW in early May 2004 and experienced the best weather!! The temps were comfortable (80s) and sunny. Crowds weren't too bad either!!
The best thing about this time is the Flower & Garden Fest at Epcot. Everything says "spring is here", from the wonderful flowers to the concerts at night. You'll love it!!