If you go right at lunchtime, you could wait an hour. The place is right by the pool and it's very small. If you want lunch there, you probably should either try going right when it opens or wait until the lunch crowd is over, perhaps mid afternoon.
Personally, I don't think it's worth a wait. It's hamburgers, hot dogs and ice cream, all of which I can get at a ton of places at home or even over at the Dolphin which is a 5 minute walk from B&C.