Yes it can be cold enough to not want to swim! However, we were there the week before Christmas last year. We swam every evening and could not wear the winter clothes we packed. I had doubled with shorts thank goodness. I would think light windbreakers and a pair of sweats to go over shorts if...