To clarify, the restaurant borders the bayou that leads you into the ride. You don't see any pirates while you are eating. However, you will hear the occasional scream.
That being said, the Blue Bayou has the best ambiance of any restaurant at the resort and should be tried at least once.