For our last cruise, my baby was probably the youngest baby on the ship. His due date was exactly 12 weeks before the sailing date. The TA probably thought I was crazy. He ended up coming 3 weeks early, so fortunately I didn't have to stress over whether we needed to induce labor in order to make the cruise! (this was back when the minimum age was 12 weeks).
For me, I had a great time. We had older daughters, 7 and 5. It's like being at Disney World, but the hotel room is a 5 minute walk away, no matter where you are in the park. I don't have to stress over where to eat, when mealtimes should be, or whether my baby is going to be totally off schedule because we were dragging him all over the park nonstop all day long. So easy to keep the baby's schedule on the ship.
That said, it really depends on your expectations and desires for how you want to enjoy your cruise. I have 3 kids. Sitting on the verandah staring at the ocean while my baby is taking an afternoon nap, while my older kids are going crazy somewhere else on the ship either with my husband or in the kids club, is an absolute luxury. Sure, I missed a show or two, and dinner had to be cut short a couple of nights, but it wasn't as big a deal because (a) we still got our money's work of entertainment for the rest of the family; and (b) it is just so lovely to come back early to put down a sleepy baby and then get to enjoy in peace a beautiful stateroom that someone else was responsible for cleaning two times a day!!!!!
If the baby had been my first and only child, I would probably be miserable, thinking of all of the fun I was missing elsewhere on the ship that I was paying a premium for.
So if this is a once in a lifetime kind of trip, that you want your kid to remember, and to maximize your own enjoyment, I would probably recommend waiting until the child is old enough to go to the kids clubs and enjoy it. If your kids hate being in a childcare setting without their parents, then the cruise may not be as enjoyable for you, even if they are older My friends took their kids who were 4 and 5 at the time, who refused to go into the kids club, and she was still stuck with entertaining them all day long. For that amount of money she said, she would rather entertain them at home.