I guess I don't understand why someone would want to do this. Maybe because I tend to be more on the "lazy

" side, but when I get back to the Hotel and want to go take a swim it would be to relax and have some chill time. Maybe get cooled off, lay on a lounge chair for a bit and then head upstairs to a nap is more my idea. The last thing I would want to do would be walk back from another Hotel, wet and sloshy all the way through DTD to my Hotel. Way too much work for me. If a particular pool was important to me I would just stay at that Hotel.
Now, I do understand wanting your family that is not staying at a Hotel to come swim with you. They only way I think it would be correct though is to talk with the front desk. Like others have said, if you are under capacity just add them to your reservation. If you are over, maybe they would still be okay with letting them swim. I would not want to just chance it as I think things like that bring "bad karma" and I just don't need that in my life.