A party for 5 is just as hard to get. Our party of 5 is seated at a table for 6 which works out great for us, because grandma can just show up as our 6th person, and she doesn't have to decide until that morning.
If you book a party for 4 then show up with the baby as the 5th and play stupid like this was your (not the CM's) honest mistake, they might let you do that. It would probably work best if your dh told them he'd be willing to give up his seat so you and the kids could be seated. I bet they'd take pity on you as long as you were very sweet with them.
I agree with them not taking up tables for 4 with only 2 people. There are way too many families with children trying to get into this breakfast not to fill all the seats possible. There are tables for 2, but they are very limited.
I imagine I'll ruffle a few feathers with my comments, but like another poster's signature says, "My opinions may not be popular, but dude I don't care."