Let me put it this way. There are stupid people in this world. While you don't have a responsibility to look out for their stupidity, it's in your best interest to take the time to do so.
It's like driving. There are idiots who will crash into you. I don't have a responsibility to look in my rearview mirror when I stop to make sure I'm not going to get rear ended, but I don't want to get rear ended.
There are idiots who will ram you with a stroller without apologizing. I don't have to turn my head a little (because, unless you have neck problems, your head should be able to turn enough for you to see behind you with peripheral vision) to make sure nobody is going to ram me, but I do because I don't want a chunk of skin missing from my leg. If you don't want to take it upon yourself to make sure some stroller wielding moron doesn't ruin your day, that's your problem.
Bolding mine.
Personally, I think both sides need to start acting like adults. Everybody should watch where they're going and apologize if they bump into someone. But that's not going to happen. So I'm just going to perform the evidently arduous task of looking over my shoulder to save myself from injury.