We can't hear our side neighbors, but we can hear our behind neighbors. It is not bothersome usually, but it can be freaky when I'm home alone, as it sounds like the noise is in my house! It's mostly their vacuum and they sound like elephants going up and down their stairs (which are against our shared wall).
More problematic for us has actually been the little sound tunnel our development has. Our townhomes face the next set and are very close. The outdoor sounds seem to magnify in the small alleyway, but this is also annoying neighbors I suppose. The old couple with multiple giant windchimes, the guy that works on his cars at 11 pm, the frat boys who play games in the street at midnight... Those are all my complaints though, it really isn't bad - not issues that make me regret the townhome every day or week.