I would pick based on which characters you want to see and what you else you want to do on your flex day. Do you want to tour the holiday decor of the Boardwalk area resorts? They are worth seeing, for sure. You can view the CR decor when you return for your cruise.
Here is my opinion about the breakfasts: Cape May is the bottom two of character breakfasts for my family. We love BC and often stay there but Cape May has fewer food options than most and I find the dining room itself borderline unpleasant (typical hotel decor - could be anywhere - and I hate the buffet set up). I haven't had brunch at Chef Mickeys yet, although I have seen photos and reviews and it looks like it is mostly still breakfast foods plus better desserts (a good thing, IMO!). My kids would pick Chef Mickey's breakfast every time. We love the classic 5 characters and watching the monorail go by. The Contemporary is just so classic.
But all that said, I would still choose either based on convenience and characters.