The food is fine. It's isn't spectacular, but fine. With dinner you're mostly paying for the experience, like you would with any character meal. If you really think dinner in the castle is going to be a better experience than breakfast, then yes. Otherwise, no.