It varies depending on the cruise. Sometimes the Captain chooses to stay late to give the crew some shore leave after dinner.
The all aboard time will be posted in the Navigator for Key West day and announced several times. You can also verify with the crew members when you leave the ship.