I think you'll be OK, we've been many times in May and never had any problems. We just got back, and I know its a week earlier than you'll be there, but the crowds were very manageable. I've also been for Memorial day about 3 or 4 times and it still isn't that bad even that late into May.
What you have going for you is that the Northern school systems aren't out until June, so you won't see mega crowds until then. That week in May will also be MUCH less crowded than Thanksgiving. We been for two Thanksgivings, and it is packed!
I also echo what the other have said, you know what you are doing, so you won't be trying to get in line for Dumbo at noon
or anything crazy like that, so even with higher crowds you'll do fine.
oh, and I thought I remember you were planning on staying at POR in the alligator bayou section for your next trip. We just got back and stayed there and I didn't realize it, but there are steps even up to the first floor. I guess they are trying to mimic the bayou architecture where all the houses are built on stilts, but it was a little bit of a pain with the stroller. There were only three steps so it was manageable, but I'm going to be sure to be in the mansions side next time.
The bugs were also really bad on that side- I guess because of all the bayou foliage. There were lots of yellow flies out, and I got bit a few times just standing outside the room, and I was panicked that they would get Everett.