We took the 2 hour historical city tour first thing in the morning after the ship docked. It was great..you get let out at all the famous places, no rushing.
Then, on our own..we went over and had a very nice lunch at conch fritters when we got back. Close to the dock, about a block or so away from the straw market.