I agree with the previous posters that said it depends on your family!
We found that the afternoon breaks were important to us. We would go pretty non-stop in the mornings at the parks while they were not quite as crowded. When we got to the hotel for a break, we would nap, swim or sit in the lounge with a snack and a little TV downtime. We would also gauge our kids exhausion level/mood and make a call as to whether our pre-trip plans would still be fun given the current status of our children.

We specifically stayed at a monorail resort despite the higher costs to minimize the travel inefficiency and to provide an entertaining environment for the downtime. My kids really loved the volcano pool at the Poly.
I specifically recall a morning where we went non-stop through every ride in Fantasy Land at least once, with favorites (Dumbo, It's a Small World) 2 or 3 times (5 times for It's a Small World that day despite my best efforts to forget this fact even in light of the fact that I actually like it

) and the last breakfast seating at CRT for a princess "lunch". We did a few more attractions as we left the park. We certainly needed the break that day; in fact, gauging the kids mood, DW and I canceled our ADR at one of the parks and made a new one for Kona Cafe for an early dinner. We hung out at the resort and relaxed the rest of the day. We ended by letting the kids stay up a little late to watch the fireworks in PJs from the club lounge.
We had fun, which is our primary goal on vacation. I think the afternoon breaks were critical in us having fun. With our kids, if we kept pushing, we might have seen more attractions, but I don't think that time would have been fun for either the kids or us.
Now, you've got my $.02 on this one. I think the breaks definitely made our trips more magical and fun! We plan to do the same on the next trip.
As an aside, during our last trip we tried out the Neverland Club on a lark, kids were not really up to more touring but were not really tired enough for an early bedtime. In any case, the kids loved it. Two nights later we asked the kids if they were ready to go to the parks after our afternoon break and they responded that they rather go to the Neverland Club. In any case, the CM concierge was able to get DW and me in to CG for dinner, and we returned to pick-up two very happy kids who had spent the evening making pixie wands, pictures and assorted other crafts.