Have you actually been during a "slow" time? The wait times are typically significantly shorter AND there are less people milling about everywhere. If they do reduce staff and run the rides at lower capacity, it isn't particularly noticeable.
Granted, I only go every 3 years or so, but there's a stark difference between Christmas week and the week after Labor Day.
New Year's week = literally people EVERYWHERE all day and late into the night.
Labor Day = most attractions having less than 20 minute waits and almost no lines at the bathrooms. The hours are shorter, but you don't necessarily have to wait in line at the bathroom.
I've been during both time periods, and while I'm not particularly keen on going back during either, Labor Day I would happily do again without my husband (due to his heat intolerance). New Year's week I would only go if either a child of mine was performing (meaning someone else chose the week) OR someone else was paying for the entire trip.
Honestly, I'm not even particularly excited about going during Food & Wine again, because the crowds at Epcot in the evenings are obnoxious.