Whether leaving and coming back will be a hassle is going to depend on several factors:
Which park? Leaving MK and coming back to the same park is going to be a hassle because you have to park and then take a monorail or ferry. I'd say add at least fifteen minutes travel time from when you leave your car before you can be at the gates for the MK--probably more. At all the other parks, the parking lot is right outside the gate.
Where did you park? To say the parking lots are huge is an understatement. You can definitely walk from the park gates to the closest spots, but I wouldn't want to do that (especially if I was tired) to the farthest spots. That's why they have trams. While they run constantly, that's going to be some wait time.
What time of year? The more crowded the park, the more crowded the parking lot, leading to a longer average time to get to your car. Also, depending on how close you're staying, if you're going in summer, the car's air conditioning may not even begin to cool your car after it's sat in the Florida sun for a few hours. (Also, if you leave during the middle of the day, and that park closes to capacity while you're gone, are you guaranteed re-admission to that park?)
Where are you going? Obviously, the further away you go during your break, the longer it will take you to get there and to return.
How long are you staying? Let's say that the time for your break, including round trip travel, takes three hours. If you're only staying four days, you may not want to take that time. If you're staying ten days, that may not seem so significant to you.