I would never do it. We choose excursions that are suitable for everyone in the family, because we view our cruise as a chance to spend time together as a family. If an excursion is not suitable for my 3 year old, I don't book it. I can't imagine that I would enjoy going off the ship without my kids anyway. I'd be constantly wondering how they were.
To each their own though.
To each their own though.