My friends and I are grown women in our early 40's and have no kids, and we enjoy Goofy's Kitchen immensely and have been eating there since 1992 (see TR - link in signature below)! If adults look at characters as only being around for kids, then they probably won't enjoy themselves. Goofy's has such a wide variety of food that I think adults will be happy. And Storyteller's Critter Breakfast has the right kind of laid back, calm atmosphere that I think adults don't mind.
Interacting with the furry characters is fun. They know their limits. They come and pose for photos and don't spend too much time on the adults if there are kids nearby. (I always give them a giant hug in pictures!) And if they see that an adult is not receptive to them, they will move on, but sometimes a particularly playful Chip or Dale will draw on our table too and sign their name for us. I love it! The only time it can get a little awkward is when the face characters like the Princesses come by, and they have to stay in character, even when talking to grown, 40-year-old women, and we always feel like we want to let them off the hook and whisper to them, "It's okay. You can be yourself and talk like a normal person with us!!!" When posing for pictures, I don't give giant hugs to the face characters but I feel like I can do that with the furry characters!!!

I think that as long as you are of sound mind and decent health, when you go to any Disney park or restaurant or hotel, you should let your guard down, let your inner child roam free and just play with reckless abandon - no matter what age you are!!