I didn't have a problem, nor did the friend I rode with. My glasses did get a bit wet at one point, but they were never in danger of flying off. I'd say, if your glasses fit well, they're safe.
If you're worried about it, you could always get a string for them, just in case.