The Boardwalk restaurants are open for lunch. That is a good time to go there because the crowds are lighter. Most of the entertainment starts to happen in the evening on the boardwalk. At night it is a very lively place with a lot going on in the way of entertainers and a small midway.
The Flying Fish is only open for dinner, and I believe that Spoodles is breakfast and dinner only. Therefore, your lunch choices are ESPN Club and Big River Grille, plus they have great sandwiches (with chips and fruit) at the Bakery.