Feel free to visit the resorts and enjoy their shops and restaurants. Use of the pools is restricted to guests actually staying at the resort.
FWIW, so many people "pool-hopped" to SAB (feature pool at the BC/YC & BCV), that Disney fenced it off and gives out wristbands to registered guests. SAB was becoming so crowded that registered guests were complaining and Disney had to do something.
While not all the other resort pools are fenced or policed quite as consistently as SAB, you may be asked to leave if you are not a registered guest of the resort.
Best wishes -