We have many non-WDW trips to Florida, plus a respectable number of trips where we only visited WDW for a single day of a longer visit to the Orlando-area. We also have many week-long onsite stays, but not a week onsite where we didn't go to a park at all.
1. We generally stay off-site if we aren't going to go to the parks. Offsite we get a much better hotel for the same price. We've done single WDW-hotel days that weren't park days. Once we even did two non-park onsite days- but that was in December, so we used those 2 days to see all the holiday decorations and to enjoy non-park festivities.
2. The Orlando area has many fine things to offer beyond the mouse. Gosh, there are like too many to list. If you are willing to travel along the I-4 corridor, than you'll find a dizzying array of exciting options. Everything from islands with pristine beaches, parks, museums, outdoor adventures, spectacular historical sites, fabulous food, world-class hotels, super dining, and even multiple high-quality non-WDW theme/amusement parks!
3. All that said...the mouse is hard to resist when we are onsite. More than once we tried to do a non-park day, but failed. (As in, we ended up doing a park day.) Onsite, well, all roads lead to Rome. (You're picturing the Rome depicted in Spaceship Earth, right?)
4. Truly, even if you aren't interested in STAYING offsite, it pays to consider offsite meal opportunities. If you want a theme park atmosphere, consider the many fine options at Citywalk. Many of the Orlando chains are the same ones you'll find elsewhere, you can also find a host of non-chain eateries that are very good, though there are also some duds out there. Beyond the mouse, Orlando is a thriving area. In general, you can eat better quality food for less $.