We've done both. Personally, we prefer Cape May for the food as well as the character interaction. It's not as "busy" as Chef Mickey's and the characters tend to spend more time with you.
I have been to both many times and prefer Cape May. The atmosphere is much less chaotic there. We also prefer the food choices at Cape May. Plus, when you are finished with breakfast you can just walk right on into Epcot via the International Gateway!