I think you should establish some ground rules, first with your wife, then with her parents. Things like: if Sunday dinner is expected each week, do you alternate hosting?
Each side should call for an "okay" before dropping by. Added to that should be "no hard feelings" if it's a bad time, for whatever reason. Keep in mind, it can't ALWAYS be a bad time--it's much more polite to say, "we're slammed tonight, but maybe tomorrow afternoon?" or whatever. So it's not avoiding them, it's setting boundaries.
You should decide if you want them to knock/ring before entering. Lots of people have close-by relatives with keys for various reasons--nothing wrong with that, but if the in-laws are 10 minutes early and you just got out of the shower--yikes!
If they're around a lot, there's no reason you should be expected to entertain them in any way, every minute they're there. A man-cave or chores while they visit your wife is fine. A lot of it is your approach--if they SEE you running to the man-cave when they pull in, that's not a good look. But if you chat for 10 minutes, then say you have to get XXX done and excuse yourself, that's fine. Heck, your FIL may even help with yard work or fixing a chair, if he's inclined to.
When our mom was alive, my sister would drop in on her a lot, partly to do laundry (Sis didn't have a washer/dryer). But--she'd always bring my mom stuff. You know, "I got a great deal on bananas, but I can't eat them all. Here's 3 for you!" It was a win-win--Mom lived alone and didn't get out much, so fresh fruit/vegetables were much appreciated. You could do something similar with Costco runs. It would show that you value your in-laws, even if you don't want to be joined at the hip.