I mean...I get it...
My wife and in-laws go out of their way to eat at a certain restaurant chain that's no longer in our area, but is sprinkled throughout the south and midwest. It's basically a Golden Corrral, but by goodness they'll drive an hour out of town while on vacation just to eat there for the nostalgia. I'd actually put Golden Corral over this place.
OP, perhaps just go to a Golden Corral closer to closing-time. Sure, the food quality will decrease (is it even possible?) but if the dinner rush is over, it can maybe be "quiet."