Other than our one trip to Junkanoo, we've always done shopping on Bay Street. My wife replenishes her cosmetics for (apparently) good prices. Check out John Bull for that. I've no first hand experience. Lots of people on the boards chat about going to Atlantis via the Comfort Inn workaround, but that's not really our speed. Our kids are teens and beyond the water park thing. You have
Castaway Cay the next day, and if shopping isn't your thing (booze is cheap too), then maybe staying on the ship may be an option.
We're adventurers and roaming in a foreign port is fun for us (I was in the US Navy, so it's not a new experience), but, once you leave the immediate port area, Nassau isn't very welcoming. It's very much a 3rd world country.