We went in 2005 in April. It was not near Easter week, just at the start of the Flower festival, and just before the start of the big "Celebration", which started in May. While we did have to pay peak season room rates, otherwise it was a fantastic time to go! The flowers in Epcot were amazing. (I understand that that May the crowds were failrly high last year, as the room rates drop and the big celebration kicked off) I would have like to have seen the talking robotic dinosaur though. Lucky, I think it was called. It sounded really neat.
This past year, we did a flukey thing for us, and took a long weekend trip in early May. Since it was a flukey trip, we stayed at ASMu to check it out and to save money.
The weather was hot, the flowers were less spectacular (due to the severe draught). The crowds were not bad, but the teens were definitley evident. It wasn't a major issue, but sometimes teens on their own don't practice great family etiquette - language, manners, line cutting. I'm guessing the value resorts were one of the more popular places for the teens to stay, as our resort seemed to have quite a few teen groups. No big deal really, but they were at the pool very late each night. Our room was fine, but I wouldn't have wanted to have had a room overlooking the main pool if I was one who goes to bed early (we aren't anyways). The teenagers were also at the pool in the afternoon, but then they were supervised by their chaperones, so they were just like anyone else really.
The weather was also unseasonably hot (even for Florida) so it was a little hotter than I like it to be, but it wasn't too bad. Usually it should be good temps. The weather in April was perfect.
The one nice thing about May was the amazingly low rates - both the airfare and the room were a steal! Overall, we had a lot of fun, and park crowds weren't bad. Still, if I had my choice again, I'd probably pick the end of "peak" season, so long as it doesn't coincide with Easter.