We did May 28th through June 5th last year. I honestly didn't think it was that bad. It was definitely more crowded than our usual trips in the off season (late October/early December), but attractions were still doable. The only thing that got annoying is that there was thunderstorms practically everyday that lasted for hours. It was a good thing we brought a 2nd pair of sneakers.
Oh and Pleasure Island was packed because of Gay Days, but it was sooooo much fun!!!
I'd go again during that time period.