Yes! I took my kids at age 2.5, 3.5, and now 4 turning 5. They never napped, but we took it easy in the afternoons and do an early bedtime (8pm), so I always booked our FPs for the morning. This weekend will be our first trip with some afternoon FP days and staying out later some nights.
I think one thing is to be ok with being flexible with normal bedtime schedules. If you are wanting to be back at the hotel by 7:30pm each night, you'll need to eat dinner no later than 5 or 5:30, unless it is at your hotel. If you are wanting to take an actual midday break at your hotel, this will not give you much time to go back to the parks before dinner, unless you are staying at one of the closest resorts to that park.
You may find that 4 year old skips naps during the trip and that 20month old naps in the stroller, then you could push through from morning til afternoon, have dinner at 4:30/5, then back to the resort for bed. I like using table service lunches as part of a midday break. Sit down for an hour or so eating and resting, then go into seated shows/rides that don't have a long wait.