I can see both sides, and basically it comes down to being considerate no matter your viewpoint.
I'm staying at the Polynesian, and don't have an issue with people coming to visit the resort....see the sights, walk the beach, or enjoy the surroundings. I'd like to do the same at other hotels on my upcoming trip. It will help me to evaluate if I'd like to stay at another resort.
If you do visit a resort, be considerate of the guests staying there. They have priority over you for using the amenities, because they have paid for it (not being elitest, just truthful). If you're staying at resort, understand that people are curious and want to enjoy the total Disney experience, which includes the resorts.
It's the abusers that get everyone upset. Those that don't want to pay for the deluxe resort, but will do everything to take advantage of them. Those that don't feel others shoud be allowed into THEIR resort....it's Disney's resort, not yours and they know best how to run it.
If you stay at a resort, you have a right to expect to enjoy its amenities before others that aren't staying there. Conversely, you also have a right to visit other resorts to enjoy the sites, without being intrusive. No matter your view, just treat others as you would want to be treated. A little consideration goes a long way.
Where you stay doesn't determine what type of a person you are or your status in life. It's just a matter of what best suits the vacation at the time.
Please remember that we are all here because of a love of Disney and wanting to share it with others. If we're considerate of each other (which I guess is a big feet these days), everything will be OK.