This was a concern of mine before our cruise as well. I was a bit uncomfortable leaving our two younger DD's on the ship while we were out of contact for a few hours. We did decide to book the excursion and let the CM's know when we would be back on the ship. Our pager went off very shortly after our return...one of the girls fell in the club and tore her lip open. The CM's handled it very well, but I felt a little guilty for not being there. As it turned out, she didn't want to go back to the club for the remainder of the trip because she was afraid we wouldn't be there if she needed us.
Even though this little accident happened...I still plan on doing an adult excursion when we cruise again next year. The chances of something happening is pretty slim...and even if something happens, I've learned that the CM's handle the situation well and they know precisely when you have returned to the ship.