We were there this past year at that time. Had never done that time before and didn't know what to expect, just like you. We had always gone third week of April, Spring Vacation, but this year only had a long week-end, 4 nights actually from Saturday until Wednesday. We found it not bad at all. We did tons of stuff. Compared to Easter week, which we've done a couple of times, it was empty! I'm actually hoping to do this again next year if our schedules allow it. I think it is all relative. If you're gone during really low times you might think it is crowded, but if this is the only time you have I say go!!
Good Luck!!