It depends on what you want to pay, what experience you are looking for and what kind of food you want and which park you are in.
Certainly Victoria and Albert is worth it, but if you are talking about leaving the MK to go eat at a hotel you probably wouldn't choose V&A because you might not be dressed like you would want to be.
Leaving MK I would say Ohana, Artist Point, Narcoossees, Citricos and the California Grill are all worth it.
At the AK, Boma and Jiko are certainly worth it.
From Epcot or DSH Flying Fish and Bluezoo are worth it and ESPN or Cape May might be depending on the mood you are in.
But if you are at the MK, would I go to Boma or Flying Fish, etc? Probably not and ditto for other restaurants that could take 30-45 minutes to get to from the park you are in.