To me it depends on how many park days you'll do.
We recommend leaving 1-2 days minimum just to hang out at the Fort and enjoy the stuff you can do there during the day. Swim, bike, explore, take part in activities, etc. Those are also the days we head over to Disney Springs to shop for a while or play putt putt. Or visit a water park.
Our normal mode has been for every day in the parks we have a day NOT in the parks. If you plan to be a theme park warrior and go to the parks every day for any substantive amount of time, you will miss out on the Fort magic. To me a vacation is meant to be relaxing. To me that means some down days.
Bama Ed