I found the food at Cape may better. I also found the staff better. Also at Cape may your on the Boardwalk. It's easy to just take in the atmosphere after dinner or a character breakfast or lunch and just do the Boardwalk. Chef Mickeys is in the Contemporary resort, which I have always considered the least attractive of the Disney top end resorts to visit.
Where you are coming from? If running out of the Magic Kingdom and you want to return there, then Chef Mickey is literary within walking distance, or an easy monorail ride. Cape May is a bus ride. If your at Epcot or Hollywood studios then Cape may is an easy boat ride from both and within walking distance of the back gate at Epcot near the French Pavilion.
I am not a big sea food eater either unless but I found items I like at Cape May.