I won't usually go back to a restaurant if I didn't enjoy the food. There are just so many other places to try where I could get better food. I won't necessarily avoid a place after one bad experience with service. Sometimes it's just one person having a bad night. I will, however, avoid a restaurant if I think poor service is part of the culture.
For example, we go to a little touristy town in the next state about once a year. We used to always eat lunch at the same restaurant. Over several visits, we saw the place just going down hill. The food was still fine. It was the attitude of the staff that was bad, and it was a continuing theme from the hostess to the server to the busperson. The hostess could hardly be bothered to seat us. The server finally brought a round of drinks. She never appeared for refills. The food took a long time to come out. We asked for drink refills, which she finally brought after another request and halfway through our meal. The people at the next table asked the busperson for a bottle of ketchup. He grabbed the one off our table without asking if we were done with it, which we weren't. It was just bad overall service. It happened a few times before we decided we were done with the place. I have to think it's a culture that's either ignored or perpetuated by the management. The food was always good, but we don't want to be subjected to that treatment when we're on vacation. There's a much better place down the street that's smaller, but we don't mind waiting for a table to get a better experience.