YES. you have to count every single human being in your reservations, for the fire code and capacity issues. Lets say you indicate its only the two of you, your spouse and yourself...and they give you a table for two. A table for two, in my experience, is always a tight squeeze. where will you put the highchair? know what i mean?
If you leave out the baby, they will count him or her at the time you arrive. So you're no longer a party of 2 but a party of 3. They will not put you at a table for 2...now you'll have to wait for a table that will accommodate 3. It's easier to reserve the party of 3 than it is to have your party size changed when you arrive.