Yes, yes yes! We take our boys everywhere, within reason of course. For instance, I would never bring them to a restaurant at 9:00 at night, way past their bedtime! The big one is only 3 and the little one will be 1 May 1st. We usually go out to eat anywhere between 5:00 and 6:30 and never have a problem. They are well-behaved and people always remark about that, in a good way of course. Well, mostly the people in our party, but still. Not many dirty looks. If we do get dirty looks, I try and ignore them. The odd (and I do mean very odd) time my older son freaks out a little at the dinner table, I will take a walk with him and remove him from the situation so as not to disturb other diners. Or at least keep it to a minimum. A lot of times also, we will take them to a nice Sunday brunch, 11 AM, family hour.
If a couple of at dinner at 5PM at a place that is very accepting of children, they should reconsider and go at a later hour. I do think that after a certain hour kids should not be eating out anyway, or be out at all!! That's just me though, I like sticking to our normal routine!