We always go Memorial Day weekend/week. This year is our 4th year in a row, it's our end of school trip.
It's not very crowded at all, as holiday's go that is. A common misconception actually.
Realize, only a portion of the country has gotten out of school by Memorial Day. The portion still in school are too close to the end of the year to take time off for a trip to Disney. Maybe some close by come for a long weekend but most of those are the ones out of school by Memorial Day anyway.
Memorial Day Holiday is really very similar to Labor Day, which is commonly thought to be a good time to go, for the same reason.
Only a portion of the country is out of school for Labor Day. A portion has already gone back to school and it's too close to the start of the school for them to take off for a trip to Disney.
Lines are very reasonable and we even find that we have many days without any rain. It's become our favorite time and why we pick going that week for our summer trip over going just before school starts back. A week in early August is much more crowded, hotter and rainier.
Last year we even went to BB on the Friday before Memorial Day and it was almost dead. It was a clear, blue sky day too, not a cloud in the sky, not too hot, not even too humid. We had a great day and very short lines all over. It was even the first day of their Frozen Summer Fun event, we met Olaf and Kristoff, no waits at all.