DW and I have done both -- we decided to avoid Breakfastasaurus (or Ugh-a-saurus) in the future. It was not Disney's finest hour for us.
Cape May, however, gets 2 big thumbs up. Yes, it is a character meal, but it is somewhat quieter than others. It is definitely less frantic than Chef Mickeys and it can be quite fun for adults without kids. Like someone else said, the characters greet you and will interact based on your reactions. So if you just want to say hi and get back to eating -- they leave you be. If you want some interaction, they'll be happy to do that, too.
Food is good and very plentiful. On our last trip (Nov.) we felt the quality was slipping just slightly. The bacon was a little overdone and the scrambled eggs a bit dry. However, they have marvelous fruit, granola and yogurt selections, Mickey waffles, and Krispy Kreme donuts!