We went to Cane Garden Bay via a not very good
DCL excursion, so I definitely recommend just taking a taxi instead. It isn't far from port so won't take long, and in the Virgin Islands taxis are generally open-air and shared with other tourists going to the same place. The taxi we took on St. Thomas cost $8 per person each way for a similar trip. Be sure to bring cash for this.
Didn't take a taxi in Tortola, but there is a tourity little area right at the port- so I imagine it would be very easy to find where to catch a cab. Tortola is safe and peaceful (no one in your face, trying to sell you things at the port, etc.) It should be easy.
Just keep in mind that while Cane Garden Bay is just fine if you want beach time (beach is clean, nice sand, peaceful, not extremely crowded), there is nothing jaw-dropping about it, either. It's just a beach, but we enjoyed ourselves as that was all that was available to us that day (all the Virgin Gorda/Baths excursions were cancelled the day we were there

). I also recommend wearing your swimsuits under your clothes & having a potty break right before departing the ship, as the bathroom of the restaurant on the beach (where we were pointed to when I asked for one, even though we weren't eating there), was filthy.