I think it would depend on your situation. Are your kids old/young enough that they wouldn't throw a fit doing a quick in and out? My DS is just two, and thankfully is not yet at the stage of throwing fits when we leave (knock on wood, goodness knows it could appear just in time for our trip), so an hour or two of just soaking in some sights and maybe one or two rides would be great.
But also, what bedtime do you have to be back for and how early are you getting up the next day makes a huge difference. For instance, if my kiddo has had a long day of travel, has a 7 pm bedtime (has to eat by 6 pm), which we need to keep because we are getting up at 6 am for 8 am rope drop... well the math wouldn't be on my side. Arrive at Poly at 3 pm, but I need to check out the room and drop off carry-ons, so let's say assuming no problems I'm walking down Main Street at 4:00 (which I think is possible but not probable). I'd have two hours of park time before a 6 pm eat-or-meltdown. But after eating for, say, 40 minutes, now I've got 20 minutes to get my DS into bed? Logistically impossible, not to mention he'd be so keyed up trying to get him straight to bed would be a joke. Nope. So let's say I eat at 5 pm. Well now that's just one hour of park time, 1/3 to 1/2 of which is walking/waiting.
That being said, we are arriving early on a Sunday in November. DS will need to nap, but when he gets up around 3 pm, we are heading to a park for a few hours. I feel comfortable doing this bc I know we don't need to be at a park the next day until our breakfast ADR at 9 am, for which we have a car and won't need to worry about planning around Disney transportation.
I know, this isn't really a straight answer. But I've done this "stay at the resort or go to a park" math repeatedly

in my case, my first goal is "avoid meltdown" *then* "go to park". There are just so many variables!