**NOTE** I'M JUST PLAYING DEVILS ADVOCATE HERE ON THIS RESPONSE
What is wrong with US? The breastfeeding mother apparently gets to decide what others CAN and CAN NOT find offensive based on the purpose of breastfeeding?
Why should I turn away from something I'm offended by? If I see someone do something "offensive" in public for any OTHER behavior I'm surely not going to turn my back to it.
Why can't I watch such a "beautiful, natural, etc. thing" such as breastfeeding if you're doing it in public?
If someone gawks at you when you're breastfeeding then they are a pervert but if I turn away because I'm "not suppose to gawk" it's because I've been told that is a "PRIVATE moment between a mother and her child"???

No one can win on this debate.
I know you're just playing devil's advocate, but I'll play along.
I think there are other things that we as a society say people can and cannot be offended by. Some are offended by the sight of a disfigured person, or an interracial couple. Should we tell these people to hide themselves because others might be offended? Or should we tell the offended that they are wrong?
As for "turning your back", that's just a suggestion to make the offended more comfortable. Generally, if something offends me, I'd rather not look at it and I turn away, rather than expect the offensive person to hide what they are doing, as long as they aren't actually harming anyone.
As far as gawking goes, why is it different when it's breastfeeding? Manners would require that accidental nudity (I like that phrase, thanks chicagodisneyfan) be ignored. If you were at a public pool, and a 2yo grabbed at his mom's swimsuit, exposing a breast for a second or two, would you gawk, or would you politely look away? Why would it be different if it's a second of two of breast exposure when a mother is getting a baby latched on to breastfeed?
As far as I'm concerned, the difference between gawking and watching a "beautiful, natural thing" lies within the watcher's intent. If I notice you watching me breastfeed, I can tell the difference. I don't mind someone watching me breastfeed if their intent is honorable, just like I don't mind someone watching me walk down the street appreciating the way I'm interacting with my child. But if they watch me walk down the street, staring at my hiney, I DO mind that. I shouldn't have to hide because some idiots can't take their eyes off my bum. Ditto for those who can't help but gawk at the inch of cleavage they might see when I'm breastfeeding.
Please note, I'm am talking about normal breastfeeding, the way it is done by 99% of breastfeeding mothers. Clothing, baby's head, and perhaps sling or blanket are covering most of the breast most of the time. I can't answer for those legendary women who take it all off and parade around topless for 15 minutes before getting their baby latched on.