For us (just 2 adults), we have eaten there twice for Dinner. Both times were during the
free dining promotion. The third was for Lunch (or early dinner

) and was OOP.
It did seem worth the cost; the food was pretty good, we got to indulge in our younger side while still being in a fairly romantic setting (in the castle), able to meet the princesses (we enjoy character meals) and got a photo with Cinderella (two professional prints in a nice folder frame).
Once we even got to watch the fireworks from under the castle! We were also one of the last to leave MK (since it was a very late reservation). It was so empty and beautiful!
With all that ... yes, it was worth it.
Now? Not really. If it is true that the photo package is being taken away but the cost is still the same, then I say no, not for us. We rarely have photopass or ride photos and having a professional(ish) photo was a huge part of the experience for us. The last time we went, the food wasn't all that impressive, nor was the service. The princesses were great though!
Granted, we've been 3 times (out of 4 visits total to WDW). I think dining in the castle, during any meal, for a major event in your life is a wonderful experience ... and one you should have at least ONCE during your lifetime
Oh and there were swords and wands involved. We took ours home and love them for decor! Ah ... memories.