Any time you need to take more than one bus/monorail/boat, you do need to leave extra time to get from one to the other and then wait in line.
The first time we ever did a character meal, it was Chef Mickey's. We were in EPCOT and I asked a CM how long it would take to get to the restaurant. She told me that it would be about 10 or 15 minutes. I have no idea which little piece of my brain said not to trust that and to get going early, but I'm glad I did. By the time we got to the monorail, realized we had to get off at the TTC and transfer to the other monorail, waited for the second one, and got to the Contemporary, it had taken well over a half an hour and we barely made our reservation time.
That said, I like the WDW transport system and we use it on every trip. (The above happened on our first trip....) If I get to running late or just want to get somewhere quickly, I'll hop in a cab, though.