It used to be much busier in the "Summer" but as Guests started to have more flexible Vacation times vs the old every school kid or teacher being out for the June - Labor Day week-end period. The big times are Christmas - New Years, Easter, and then Spring Breaks (multiple blocks of dates in March, and early April).
In addition as
@gottalovepluto indicated blocks of Fall also have much larger crowds. October bank holidays, fall school breaks, and early November for Jersey week / Veterans Day Holiday especially when they overlay.
All this periods don't mean that every week during the summer will be a ghost town, just periods of lower Guest visits between spikes of very busy dates.
Dave