Agree that if you don't want a big meal, skip Cape May! Given that your hubby likes sports, I'd go to ESPN for apps.
However, there are other options around the BW and YC/BC area depending on your mood. The lounges at YC/BC (Martha's Vineyard at BC and Ale & Compass and Crew's Cup lounges at YC) have apps and drinks. Ale & Compass restaurant is also available for apps or a light meal. You can also get apps at Todd English Bluezoo Lounge at the Dolphin. Same at Shula's lounge (as well as soups and salad).
You will likely be STUFFED after lunch at 50s PT. We had an early lunch there a few yeas ago and a late dinner at Artist Point the same day. We were so stuffed that we just split apps at AP. It was a lovely experience, but your thinking is correct about going somewhere for a lighter meal after a 50s lunch.