Last year we were there from the Tuesday before Memorial Day Weekend to the Wednesday after, and we saw a significant jump in crowd size from Thursday to Friday, with the heaviest crowds on Sat, Sun & Mon. We opted to do a non-park day on Sun just because the crowds were so heavy. Saturday we were at Epcot, which wasn't too bad, but they did run out of fastpasses for everything very early in the day. When we tried to get a fastpass for Test Track at 10:30 am, the return time was for 9:45 pm! Monday we did AK, and we were tired of fighting the crowds by 2 pm and left the park - really a bummer, because we did enjoy AK a lot, but it was just too packed for us.
Now, I've also been to WDW during the week between Christmas and New Years, and over 2 spring break weeks. The Memorial Day crowds were nowhere near as bad as the Christmas crowds, like a previous poster said. I did thing the crowds over the actual 3 day weekend were heavier than the spring break weeks, however. I don't know that I'd go again over that weekend, unless I was really geared up for crowds for those 3 days.