As one of just two adults, and have had a very special dinner at Yachtsman Steakhouse with children at the very next table, I say BRING 'EM!
The restaurant is very nice and upscale and the kids are very well behaved. If not, just take them out of the seating area and you'll be fine.