We have sailed 7 times and I couldn't care which side I was on. At each port it all depends on how the ship is docked, forward or backward....so technically it can change all the time. For instance...sometimes the Captain pull into CC forwards, sometimes backwards. In Grand Cayman you dock in the harbor so you have a lovely view all around. Sometimes you need to tender in St. Thomas as well.
Port or Starboard would be the least of my concerns when sailing. I would be more concerned having a cabin under a nightclub or something.
MJ