It's been a few years, but we have gone twice during that week as our schools finish at the end of May. It wasn't a "low season" by any means, but definitely more manageable than it gets later in the summer.
In fact, both trips, I noticed a very significant increase in crowds on the last couple of days, and the day we left to go home in 2010, I saw the crowds lined up to go into MK from the monorail and was actually glad our trip was over. We certainly hadn't experienced anything like that earlier in the week!
Since our other trips have been at spring break or Christmas, those were actually our least crowded trips we have taken and I would love to get back at that time of year again. Our schedule just hasn't worked out for it recently.