The owner did you a favor by seating you without your full party there. DS arrived 35 min later. They had no idea that DS would just order a cinnamon roll and not something hot that needed to be prepared & cooked. He still got a full 30 min to eat the cinnamon roll and finish off DH's steak and eggs.
The server checked on you twice starting at 12:10 and even asked if MIL wanted more coffee. As a PP said, had she said yes, there would have been no problem with your party lingering.
Yet, you complain that they ask you to please be mindful that there are others waiting? Did you stiff the server on the tip for that? You haven't mentioned what you did about the tip.
How about being gracious and thankful about being seated before your full party was there, instead of the two elderly people being uncomfortably squashed in the waiting area? How about being mindful and considerate that there are others waiting? Perhaps another party of 5-6 arrived right before DS arrived. They would have actually been seated first, had the owner made you wait for DS before seating you. You don't know how long another party of 5-6 may have been waiting for your table.The restaurant obviously didn't want to turn two 4-tops into another table for one party of 6. It may screw things up for the other waiters, depending on where their sections end. One waiter may lose a whole table.
My suggestion is: next time pick a restaurant that has a more leisurely atmosphere, since you know you have 2 elderly people who will take longer. Not all restaurants have a wait line. As others have said, some restaurants have such bad service that you end up taking quite a while even when you didn't intend on that happening.

Then no one in your party will feel they are taking too long, or feel they need to rush.