If you stay at SOG, you may only swim at the SOG pool. Some resorts do check ID; I've seen security ask people for their resort IDs at the WL pool, and I've seen them ask folks to leave if they can't produce a WL/VWL ID.
The resort pools are simply not large enough to accommodate people who are not staying at the resort.