Most of the time, extreme weather makes it more memorable. Except for stupid hot & humid out....
I distinctly remember the day that it downpoured as soon as our train left the station on Hagrids. Couldn't see or hear much, with eyes squeezed tight against the rain at 50mph. Got to the dry respite of the cave, poised to go back out - Hagrid says "hit the purple button," and from the sidecar I said "Noooo!".... but friend hit the button and we launched back into the rain. We exited soaked right as the rain let up. Friend went to buy a dry shirt, I just hopped on the log flume because I couldn't get any wetter. Much more of an experience than the normal dry version, and am glad we "got" to do that. Although if that ride was your first time, it would have sucked.