Our family did this a few years ago with 4 kids from age 3 or 4 up to a teenager. It was fun but something that you would only do once because it was set up like a treasure hunt and once you know the answer to the clues it wouldn't be fun a second time.
23 months might be a little young. I would consider a few things before I made up my mind. I hope lots of people give you their feedback. Would you have to pay for the baby? it was a fairly pricey little activity for our group of 6 AND I would suggest definitely use a stroller- maybe one that they can climb in and out of easily in case they want to do some of the silly follow the leader things like the rest of the group- or you will wind up carrying him/her for hours.
We were not alone. We had another smaller family participate with us who's children were young "gradeschoolers". If I remember correctly, there is a limit (12-15 persons?) to the number of guests that take the tour at once (one CM/group).
The day we took the tour was very hot and humid even early in the morning and that made it a little less enjoyable than if it had been a wee bit cooler. We had a few quick breaks and opportunities to get a drink of water.
Overall it was a fun experience, providing you are willing to abandon any self-consciousness you might normally have and fully participate in all the silliness. At the end there is a fun "surprise" character meeting for the kids to enjoy in a private unrushed setting, photo-op, bring your camera.
As the group broke up our CM commented that our teenage son was a good sport throughout and that she always worried when she saw a guest that age in the group because, well, you all know how teenagers can sometimes be, and this is definitely an activity where you call attention to yourself!