Considering the cost of party tickets, most people would recommend that you eat before the party instead of spending party time in a restaurant because there are so many party exclusive activities to enjoy.
You could have a late lunch or a counter service dinner around 4:00 - 5:00. For a table service lunch Liberty Tree Tavern or the Plaza are good. For counter service lunch our choice would be Be Our Guest.
You could also have lunch or a very early dinner at one of the monorail resorts.
If you're interested in doing meet and greets with rare characters, people start lining up around 6:00 so that's another thing to take into consideration when planning meals.