I would not go to any of the current character dinners if there were no characters, except MAYBE Crystal Palace (enjoyed the food at the dinner there very much). While we found the buffet food to be good at character dinners we went to, we really went solely for the characters and we knew the food would at least be decent so it was fine. We wanted our children to enjoy the character dining experience.
I will add that the other buffets (non-character) we went to were excellent. These were Boma dinner and Biergarten dinner. We would definitely go back to Biergarten, and the only reason not to Boma dinner so quickly is because DH and I got bad heartburn that night (NOT the restaurant's fault...we have sensitive stomachs and are not used to those kinds of spices).
I will also add that I would still go to the breakfasts if there were no characters. We enjoy a good breakfast buffet and we enjoyed both of the ones we went to (Chef Mickey's and Hollywood & Vine).