I think it depends on whether or not you want to see Spectromagic and/or Wishes and what time they are going on that night. We are going next week and have an ADR at the Castle for 5:50and Wishes is at 8 (no Spectro that night). We've done this before and it's worked out great. No rushing through dinner, enough time to walk down in front of the castle and find a spot for Wishes.
If you aren't interested in either Spectro or Wishes, 7 pm sounds like a great time to dine.