Other than a chilly dash from the pool to your resort room, you should be able to get some swimming in.
A few years ago the kids and I swam while DH did Richard Petty driving experience. I think it was about 65 degrees out. People were sitting around the pool wearing winter coats. When you come from Wisconsin, 65 degrees in the middle of winter is very warm. We had great time!!