Don't know what your budget is, but you could also look into the Fort Wilderness Cabins.
They sleep 6 - the bedroom has a queen bed & a set of bunk beds, & the living room has a pull-down murphy bed. Plus, there is a full-size kitchen & a deck outside. (There's full maid service & complimentary daily newspapers as well.)
We have 2 small children, &, while we would "fit" in any of the hotel rooms, we chose the cabin so that, after the kids went to bed, we could still move around & watch TV w/o waking up the kids.
The Fort Wilderness Resort & Campground is also a boat ride away from the Magic Kingdom - for this trip (our kids are 5 & 3) proximity to the MK was a big factor for us.
We looked at the All Star Resorts because, at first, we thought the kids would like the "themeing" a bit better, & we also looked at the Caribbean Beach Resort & the Port Orleans Resorts, but, in the end, the closeness of Fort Wilderness to the Magic Kingdom & all the "extras" at FW made our decision easy.
The Fort Wilderness area has tons of fun things to do for the kids - playgrounds, pony rides, hay rides, petting zoo, nightly campfires w/ Chip & Dale, nightly Disney movies outside, a beach where you can watch the Electrical Water Parade, wildlife, etc.
The only 2 downsides that I can tell are the 2 pools - both "unthemed" & the internal bus transportation.
Fort Wilderness is a bit pricier than 2 rooms at the All Stars - which is what we would have done even for just the 4 of us - the value resort rooms look small - I think you would want to at least consider 2 rooms at the All Stars w/ connecting doors - then it would be a like a suite - 2 tvs, 2 bathrooms, &, at night, the kids could go to bed in 1 room & you & your DH could stay up in the other. Or, take a look at the Port Orleans Resorts - which one has the trundle? The grounds look lovely, and, from what I've heard, the pools are really nice too.
Also, we got a discount at Fort Wilderness by booking our package through
AAA.
And, this is our 1st trip to Disney since my DH & I were on our High School Senior Trip in 1991, so it's been a while! But, in answer to your question - "Is there a difference between staying on-site & off-site?", I would say a definite yes. On-site you get to take advantage of Disney transportation, free parking at the parks, & the extra magic hours at the parks. Plus, all that extra Disney magic!!
Good luck w/ your decision!!