We did the sea lion swim at Coral World last month. We booked directly with CW and not the Disney excursion.
You will definitely feel safe getting off the boat and getting a taxi to CW. The taxis are literally right off the boat. You may have to wait a few minutes for them to get more people but just ask how long they are waiting. It was $9 pp so for 3 of us we just gave him $30. When you pay him, ask him for a coupon for CW. It's a couple of dollars off.
When you pay at CW you will pay for the admission gate only. You will be instructed to go to the white thing (its on their website) and pay for the sea lion swim there. They will give you instructions on when to report back. PAY CLOSE ATTENTION!! If you are late they will start without you and you are out your money. There was a family of 5 that was supposed to swim with us but they were 30 minutes after the reporting time and 15 minutes after it was supposed to start.
Anyway, CW is a cute little marine park. You can feed sting rays, see shark feeding demonstrations, lots of fun things. Look on their website. It is also next to Coki Beach and the wrist band they give you permits you to come back all day. We didn't go to the beach.
Our swim was scheduled for 1 p.m. We got off the boat when it docked and went right to CW. We paid, then went over to pay for the sea lions. We spent the next couple of hours just slowly enjoying the exhibits and seeing the different programs they were doing - sting ray feeding, shark feeding and sea lion demo. We then did our shopping, ate lunch and went to change for the swim. We reported back to where we had to swim and then waited for the other family.

Like I said, they missed it and we had an AMAZING time. After, I went to pay for the CD of pictures. You can buy it in advance. Ours was $39.99, it depends on the number of people in your party but that is the most expensive. I think for 2 it would be $29.99.
Anyway, after the swim it took about 10 minutes for them to burn the CD. I think the morning group was told it would be an hour. Maybe if they have to separate families it may take longer.
Anyway, after that was done we went outside to the slew of waiting taxis and caught it back to the ship. We did a little touristy shop on the pier and were back on board with an hour to spare.
If you want to do the swim you need to book it in advance and as soon as possible. They take a credit card number over the phone but don't charge it. You pay when you get there. And yes, the cost was minus what we paid for admission.
I understand they only have the swim two times a day, 10 a.m. and 1 p.m. It was definitely worth the money. If I can figure out how to post pictures I will. We had a great time and I'm more then willing to answer questions.