I am also blind without my glasses and I always leave them on. I have never had a problem. You could always wear a sports strap if you're worried.
How well do yours fit? One test is to look down at your feet and shake your head vigorously. If your glasses don't move, you're probably good to go.
One other thing I do is bring a backup pair of glasses with me, even an older pair, just in case something should happen. (I leave them in the hotel room- hopefully DD could lead me back there! ) Mine is not the kind of prescription that could be filled in an hour so if your lenses are also special order, you might want to consider having a backup on hand.