We were there in May (10-18) and I'd have no problem going back during that time. The weather was perfect except one time, it rained just enough to get everything wet but after that, clear sky.. the crowds were fine, if we had to wait it was for 5 or 10 minutes and that was it. The only bugs I saw were in the Animal Kingdom and I stood in line to see their show... I think if you head for WDW the first 2 or so weeks of May, you'll have a great time.
BUT REMEMBER.. the last week is Memorial Day and the place has some of the highest attendance days during that holiday so make sure you check the dates when you do your planning.. in 2006 Memorial Day is 29th so from the 25th or so until after the holiday, the place is going to be packed.
Good Luck...