We've gone the past few years the beginning of July. Two years ago we arrived on the 5th. We would be at the parks at opening and was able to get a lot done by noon (when the crowds really picked up) without much of a wait. We would then take a break and swim, nap, mini golf, rent boats, etc. We would head back to a park at night, it would still be crowded (60 minute or more waits at popular attractions) but we would take it slow, check out the less popular things.
When we went this past July we arrived on the 14th. We asked the CM's about the tour groups (who can be loud and rowdy) and were told that every year they start arriving the second Sunday in July. South American schools get out the second week of July for a 4 week holiday.
You said you are planning the 5-20. That would give you time to take it at a slower pace so the heat won't be as draining.