Last year we were there the last week of April, the year before the last few days of April, first few days of May and in previous years mostly within the first 2 weeks of May.
We enjoy going to WDW this time of year because of the lower crowds and the weather is still bearable. Begins to heat up but not like the summer months-still a little more comfortable as far as the stickiness scale is concerned. Although we have had some years where it rained a lot and even one year where it was REALLY hot but still lower humidity.
The spring break crowds are definitely gone as long as Easter falls earlier in the month. The only thing to keep in mind is grad nights. Over recent years we have been seeing more grads mixing in a few days before and after the actual events. They also stay at the resorts on property and sometimes hang in groups during the day in the pool areas.
Also if you are staying in a resort near MK expect to hear fireworks at midnight. If you're staying at WL or CR you'll definitely hear them or even see them if you like (from your room/resort).
Last year I didn't realize they moved the events to HS and we stayed at the BCV. The first night we went to bed early and the fireworks woke us all up because they were very close to the Epcot resorts. We could see some of it which was cool but it would have been nice if it was just a bit earlier.

Plus you may hear the grads walking thru the halls during late hrs/early am hours if they're at your resort. Even though many of the kids are bused in, many stay at the resorts too.
Still the pluses of going this time of year outweigh any of the inconveniences.

We'll be there the end of next month too.