I've had one since the early days when only the 3 (or was it 3 1/2?) qt was available. Even when it was only the 2 of us I always wished for a larger one. Whole chickens & roasts didn't always fit too well (esp when the roast was still frozen) and there never seemed to be enough room for enough carrots/potatoes.
I bought a 5 qt at some point, but still had trouble with those frozen rump roasts fitting. As soon as the ovals came out I got one of those....but back then it was only 5.5 qt.
We eat leftovers here. No one ever complains about them. Perhaps if no one will eat the leftovers & you won't use it when you entertain, a smaller one would work for you. The thing is, they last a long time & your boys are small now. In another 5 - 10 years, your crock should still be going strong & let me tell you about how much teenage boys eat!! (especially meat!)
I've always had the Rival pots with the stone insert. (Well, the 1st wasn't removable which was a pain!) I love them! My sister & mother have the type that is a heating element that looks like a try & a Teflon coated metal "pot" that sits on it. I don't think things turn out nearly as well with theirs. I'd never have one. Things seem to be drier & burn more easily with theirs.
JMHO - Good luck!