You'll get very mixed answers to this question...
Disney does have a sign up saying that the beach for exclusive use by Poly guests only, however they only tell other guests to leave if it's a really busy, peak time of year (NYE, 4th of July, etc). I personally don't think that you should use the Poly solely to watch the fireworks and nothing else. Take advantage of some fo the great restaurants while you're there too. We plan on spending our first evening at the Poly, and will eat at Kona Cafe, and watch the fireworks from the beach, before returning to our resort (PC).