Back, a number of years ago, we were staying at the Polynesian. When we would go to the main pool, it was sometimes hard to find a seat/lounge chair close to the pool. Most were either occupied or had 'belongings' on them. There were a few times that we actually overheard people talking about how nice this pool is and that they should actually stay there some day! This was from a group of about 10!!! There was another family that was chatting about how nice it was to just put their suits/towels in a locker at MK and then head over to the Polynesian midday for a swim, to cool off, before heading back to the MK....then one member of the party asked where they had parked. 'Oh, right here, at the Polynesian. It's free so we don't have to pay at the main lot.....you only get free parking if you stay at a Disney resort.'.....geez Louise!!! On several occasions I saw non-guests escorted out of the pool area!!!
Happens at Boardwalk as well. It's not nearly as hard to get in there as it is over at SAB. So, there can be people that are at either Epcot or DHS that like to cool off, so head over to Luna Park. And yes, I have seen people asked for their room keys and then asked to leave.
It is seldom an issue at the moderates and value resorts. They aren't as easily accessible from the parks, and to be honest, the pools have less appeal than those at the deluxe resorts. So, if you're staying at a deluxe and you have friends at a moderate, perhaps you can head to the moderate to allow the kids to swim together.
I know many say 'Oh, it's just one person, what's the harm?'. But, if 50 people said that, well, you can see what the harm would be!! At least 50 non-guests in the pool!!
When asked, the lifeguards will take each request based on the current situation. Sometimes the answer may be yes, but it can also be no.
Oh, and those DVC hoppers? Many of us feel it's more work than it's worth!!! I don't really like packing up my stuff and trekking over to another resort just to use their pool. Besides, the really good ones are off-limits to us as well!!