I've been to many beaches in the Bahamas and I think Blue Lagoon Island is the best. If I were you, I'd give it a try, even though it's more expensive than having a taxi take you somewhere. At Blue Lagoon, there are many shady areas where your baby can snooze while the older kids play either in shallow or deeper water. As I recall (but check this out), there are several times during the day that you can return to the ship-- you don't have to stay the entire day. And the good thing about an excursion is that Disney takes care of the logistics, which is just one more thing you may not want to deal worry about.
Here's more I found on the www
"Blue Lagoon Island Nassau Bahamas offers the most unforgettable encounters and captivating experiences; you have the opportunity to enjoy complete relaxation, magnificent vistas, or exhilarating adventure. Escape to Blue Lagoon Island and see the white sandy beaches and clear waters that made the Bahamas legendary.
Amenities are abundant on Blue Lagoon Island; there are gift shops, changing rooms, fresh water showers, and the Sea Garden Grill restaurant. Live entertainment sets the mood while on the island. For those of you who have relaxation in mind, why not lounge in one of the many double hammocks or take a historic nature walk."