I take mine off to ride roller coasters, that's pretty much it. It will get really cold indoors, so I'll take it off.
Don't expect this thing to work miracles, it will make you feel 5 to 7 degrees cooler, but is there really a difference between 95 degrees and 100 degrees? If you fill it with COLD water it will work much better.