Way, way too cold to swim in January if the pool isn't heated.
And I'm from Minnesota.
We tend to go December / January / February and it's not uncommon to have evenings in the high 30's low 40's, many days have highs in the 50's or 60's. Unheated pools quickly fall below 60 degrees, which is cold enough to introduce major shrinkage.