I've been to WDW over Memorial Day for the last several years. While the weekend is crowded, the week before or week after is not. On a scale of 1-10 where 10 is the most crowded, the week before Memorial Day is a 4, and Memorial Day weekend is a 6 or 7.
Happy planning!