It's definitely not as loud as any of the other character breakfasts I have been to. If you're looking for a more low-key character breakfast, then yes, this one is great for adults. I think though that all the character breakfasts are fine for just adults and would go even if I didn't have kids.
Crystal Palace also has an excellent character breakfast, with delicious food, great service, and it has a beautiful atmosphere as well. Our favorite character breakfast of all is 1900 Park Fare (in the Grand Floridian). They had some more unique foods on the buffet (lobster eggs benedict; bananas foster syrup for the french toast for example) as well as traditional breakfast foods, the food was all very delicious and the restaurant is beautiful too, as well as the Grand Floridian lobby!
If, however, you are looking to see the "classic" characters, Cape May will give you Minnie, Goofy and Donald. Another option for the classics would be Chef Mickey's, which has Mickey, Minnie, Goofy, Pluto and Donald. We thought the food was excellent there, but it is definitely louder and has more excitement than Cape May. To me, Chef Mickey's is the perfect character meal because I enjoy all that excitement that comes with it.
As far as the food at Cape May, it is delicious! Service was also excellent! Again, it all depends on the overall experience you are looking for, and which characters you would like to see, if that specifically is important to you.