I agree with so many of the posts here and became teary eyed by a couple.
My DS rides a special needs bus. Actually, it is the same size as other buses. It just has a big handicapped sticker on the front and back. Sometimes, if the bus needs repaired they will bring the short bus.
Really, the only circumstance I think people should use the word retarded is when they are speaking technically of a person's mental capacity in the proper forum. It may have a definition that one could say allows it to be used justly in other situations, but I have heard that same argument for nearly all racial slurs. I view this as the same thing. Obviously, it is not racial but aimed at belittling a group of people.
I too hate the way Last Comic Standing uses the little bus to transport their comedians. (However, of the few episodes I watched this year, I loved that they showcased the guy with CP, who was really a terrific comedian).
My son is "mentally retarded". His latest IQ test score came in recently:57. (I know many kids with autism have inaccurate IQ scores because they don't think like a standardized test expects.) So, when people demean something or someone by calling them "retarded", it says a lot about how they view my son. Is that overly sensivitive? Maybe. But, I am on my own private mission, albeit small in scope, to get people to stop making fun of the "small school bus" and using the word "retarded" as an adjective meant to belittle or insult. I kindly tell people that I realize they hear it all the time and it has become so common to use that it seems OK, but it isn't. I explain that my son is autistic and MR and that it deeply offends me. I continue that I hope they will consider that before they use the word again.
Another show that in the past (although, I don't recall it recently) that has regularly used the phrase "the little bus" as a negative is The Daily Show.
A few months ago, a local radio talk show host used the word autistic that I had never heard before and was floored. He was talking about his computer and getting it repaired. He described the computer's problem as "it went all autistic on me and couldn't communite with the outside world". I wrote to him, his producer and the station manager. If people start using these terms, esp. those that are heard by large numbers of people, it becomes acceptable. I did not recieve a reply form the talk show host, but I haven't heard him use the word since, either.
Getting to the neighborhood issue. I totally understand you. We live in what would be called an upper middle class neighborhood. Many people around here, want to portray that they are something they aren't, IMO. When we moved here, I took my son to the organized play groups and no one would play with him and no one would speak more than a few words to me. We now have a few friends here, but just as many will stick their noses up in the air and pass by. After 5 years it is getting better. DS is a very likable boy and is somehow sort of popular at school. Almost everyone knows him and everyone speaks to him. DS went to a summer day camp (with an aide) that is mostly used by people in this community. I picked him up early one day and to my delight without prompting from anyone about 50 kids shouted good-byes (including his name).

He rarely plays with kids from the neighborhood. But, last spring a mom called and asked if he could come play at their house as her daughter was begging her to invite him. I tried really hard to stay calm, but I couldn't hold it in. I started crying on the phone. I explained to the lady who I never met that it was the first time my son has ever been invited in such a way to play with another child. Ended up the other mom crying, too.
Most of this happened once my son started school. I think his school does a good job of helping him fit in and encouraging diversity. Maybe the school or a teacher could break the ice.
Enough of my mini-novel.