We are from a town with 351 people and we don't even live in town. I can do crowds for a small period of time. My DH has to have a Gin and Tonic with valium each morning we are at WDW.
So we can go when the crowds are heavy. No holidays or summers. I don't like that much heat.
Last time we went last week of April/1st of May. The weather was perfect, no too hot and we did run into crowds. On one day MK seemed almost empty!