I also use a tripod with a quick release plate...so taking the camera off and putting back on is a breeze, therefore I do take it off all the time. I will sometimes leave my tripod legs extended, and just compact them together to carry the tripod to a new location if I know I'll be setting up just a few hundred feet down the path again. The only time I'll leave the camera on the tripod in a Disney situation is when I'm moving around in a small space taking multiple photos - such as in a pavilion in Epcot. I won't bother to take the camera off to move 30-50 feet. But I will take it off to move to the next pavilion.
It shouldn't be any real problem to leave the camera on the tripod - the act itself - but you risk bumping into things or dropping, or if you have a heavy lens mounted which is unsupported, maybe stressing the lens mount. Otherwise, it's just preference I suppose.
When I'm out birding, I use a very large lens - a 200-500mm. Because of the size, I don't bother taking the camera off the tripod, because it's actually easier to carry the whole rig with the camera mounted (actually, with that type of large lens, it has its own tripod mount, so the camera body is actually attached to the lens which is attached to the tripod), the tripod legs extended but compacted, then just hoist the lens over one shoulder and grab the tripod legs at my chest with my hand in the neck strap as a safety loop. I'll walk miles with the lens like that - so I can sling it off and snap off a shot monopod-style with little notice.