I talked to person at the health club, thinking that was near the pool.
I stressed that we were staying on Disney property, but not at Swan / Dolphin. I asked if there was a day rate, perhaps for the health club and the pool. He said they do not use wrist bands, that he could get in and use the pool. I asked for clarification, my husband likes to swim laps, could he really come in and use the pool? He said something I couldn't quite understand, so I asked back if that would be cheating? He repeated back what I didn't understand, that the pools are heated. And that yes, he could use them. I was very surprised and told him that he made my husband very happy.
I will report back in two weeks what happened.