I'm getting ready to stay at the Poly and I've heard a few different things. I have heard that there are a lot of guests that are extremely against resort hopping, meaning taking advantage of another resort's amenities. I've also heard it's not allowed to visit their beach unless you're a guest at that resort. We were staying at the CR for our honeymoon last year and I thought visiting the Poly to watch the fireworks would be fun until I saw that.
I'd call Disney and see what their rules are only because I've heard so many things.
For me, since it is an expensive resort, when I go and want to visit the beach, if I were to find it full to where I couldn't enjoy it and it was because there were a lot of outside guests there, I'd be frustrated.