I went during these dates in 1999 and it was not crowded at all. We actually stayed 5/19-24 and in 6 days we saw every attraction. It was, however, extremely hot and humid, so definitely bring water with you. The lines for drinks were so long that they often ran out of certain products so be sure to bring your own .
We were @ Disney May 20-27 2002. The beginning part of the week was excellent. Light crowds, pleasant weather, and the parks were open pretty late. By the Friday before Memorial Day things changed. It got a lot hotter and the crowds were there. The Magic Kingdom was packed. If I go that week again, I will leave the Thursday before Memorial Day. Hope this helps!!