I presume you are already going to be spending a good amount of time with this friend, right? There isn't anything wrong with saying, "I've been there, and I really don't care for Cape May, but since I know that you really want to eat there, it will be no biggie, and I'll plan to eat someplace else on the night you want to go there, and we can meet up later." As long as you really mean it will be no big deal, and say it in a positive, non-guilt inducing tone. It kind of puts it back in your friend's court then if they want to offer up an alternative so you can eat together. I would hope that most adults can accept doing some things on their own if it doesn't appeal to others in their party. It wouldn't bother me a bit, and I wouldn't be having any fun with someone at a restaurant that made them miserable.