Our experience in January at VB is that it's usually decent swimming weather (of course, considering that I am from Maine where swimming weather is any time the pond isn't frozen...). We have been twice in January in 4 times in Febraury, and only once was it too cold to swim (Jan 2005, not that we didn't swim anyway).
As for Tea, you can get a cup of tea up in the Green Cabin Room, the bar up on the second floor, and sit overlooking the ocean, or over by the fireplace. I don't know if they have tea at Bleachers by the pool. The Green Cabin Room is a jewel, and a great place to just relax for an hour any afternoon. On weekends, there is often live music at the Green Cabin Room, and that can be noisy, so take that in to consideration when considering room assignments.
Also, if you are a Tables in Wonderland holder, remember that your TiW discounts are valid at Shutters and the Green Cabin Room (and possibly Sonya's, but I'm not sure).
If you are adventurous and it is available, the Kayaking expedition is excellent. It is physically demanding and will take the better part of the day, but it is spectacular. Otherwise there are plenty of other great things to do at the resort. My wife and Mom tried the yoga morning class the last time we were there (and will likely do it again next month) and found it to be very good.
You will need your room key to get in to the pool. as it is gated, like at Kidani. The beach is very nice as well (tho tough with my son's wheelchair - he does use their beach wheelchair when there and enjoys being able to join the rest of us for walks).
We love VB and try to get back every winter for a break. It is a great spot to relax and recover from your everyday life. Enjoy!