No one expects silent. I do not see anyone stating they expect absolute silence except at V&A where little children are not permitted anyway.
However, if you have children who aren't likely to sit still and want to run around the restaurant, sit on the floor, throw things or bang the silverware, or you have kids who will scream uncontrollably for a long amount of time, an early time is best when there will be more numerous families present. Flying Fish is not a quiet restaurant, so normal talk or laughter by children OR adults is not an issue, but it's not a character buffet either.