We went with a 4 month old and it was pretty easy. I think we had more luggage due to her gear, diapers, etc but she rode in the stroller, sling or bjorn and was very content to just look around and take everything in. She also sparked a lot of conversations as she'd stare at people and smile--such the flirt.
I was breastfeeding (and not very coordinated about doing so in public discreetly) so I was always looking for an out of the way bench, ducking into the baby care center or feeding her in dark shows (I'd sit in the back row of 3-D movies, etc and feed her).
Having a baby along made us slow down a bit I think (although in addition to the baby we had a 3 year old and a grandma on a
ECV so who knows what really slowed us down). But slowing down was not a bad thing in my opinion.
My hubby refused to lug the whole travel system stroller along, so we made the Joovy Caboose work (I added an infant head pillow to baby was more comfortable). In hindsight I think an umbrella stroller would have worked better--made a comfy little sling type seat for her.
I did have one moment where I second guessed our decision to bring a baby baby along. I was carrying her in a mob of people waiting for Spectro to start. I got a little scared because it was just jam packed and I felt like this was the sort of crowd that someone could get trampled in (seriously--people were getting pushy and I was just stuck in the middle, in front of the castle). But now I know to be more aware/avoid those situations better.