Don't worry...
First, on the cruise, your room is always within walking distance, so you can take him back there at any time without any major complications.
Second, he may surprise you. Try to keep a couple of his favorite things with you. Ie, a blanket, stuffed animal, pasifier, or snack...