We do it a lot. Honestly, the resorts and those amenities are what keep me coming back - the parks hold very little interest to me as of late. Stale, etc.
We typically stay at a WDW hotel...AKL being our favorite. There are so many things to do THERE that you can easily pass your whole day.
We also like the Boardwalk area - beautiful, we get ice cream, people watch, etc.
DS is fun, too. Lots of little places to pop into, Splitsville, catch a band/act at the bandshell, Christmas shop for that years ornament, bench-riding, people-watching, good meals (EoS is still a fantastic bargain!).
We hit Fort Wilderness one morning for breakfast, and the explore - the barns, watch them shoe horses, play a few games of checkers, hang out on the beach for a bit.
Swimming, of course.
Mini golf a few times.
Most of the resorts have their own things that they do to have fun - Yacht and Beach still has trivia some nights, I believe? Arcades everywhere, fishing, boat excursions, fireworks cruises, hidden Mickey searches, wagon/carriage rides, water parks, spas, a fancy dinner out to celebrate health, etc.
There is plenty to do and see that would fill your days...even away from the parks!