I have been 6 times during May in the last 7 years. The crowds have been easily managable though I usually am gone by the end of the third week so haven't dealt with groups stretching Memorial day into a long vacation. The parks are fairly crowded on their early entry day and the MK is packed on Saturdays. If you hit each park on a non early entry day you will be fine. Sometimes the Spectro Magic Parade and Fireworks are only on Saturday at the MK. Be prepared for huge crowds and a long journey out at the end of the night.
The weather has varied quite a bit. Two times it was in the mid 90's during each day with beautiful warm evenings. 3 times it was in the mid 80's with an occaisional day in the low 90's. One time it was in the high 70's to low 80's the whole time. Rain has been scarce. A couple trips it didn't rain at all. The rest of the trips it rained once or twice for 20-40 minutes and that was it.