May is our favourite time. Have gone in May about 10 times. Tried December, to cold to swim. Tried late January early February, short lines, but too cold to swim. Going back in May, the only problem with May is, it's becoming a popular time to go to the World.
We like it because, it is regular season (cheaper than peak), the lines (in the past) have been reasonable, the weather is pretty reliable (the odd rainstorm, usually once during our trip), and not a lot of humidity, we enjoy May. Have fun whenever you go, remember to enjoy the magic. Have a blast.