frann, you should check around with either a mortgage broker or some banks to see if it is worth it to refi. With interest rates so low, you might be able to refinance to a 15 year loan (with rates lower than a 30) for the same payments as you are making on your 20 right now. It's hard to say without running the numbers, but it could be worthwhile.
C.Ann, the concept of a "starter" home isn't so ridiculous. A couple just starting out has different needs than one who has kids. When you are fresh out of college (or just a few years out), you generally are making the lowest income of your career and have little cash to put down so it makes sense to buy a smaller home (2-3 bedrooms). Once you have built some equity and your income improves, most people start to think about having kids and that 1100 sq ft house can feel pretty crowded with children, so couples will upgrade to a "family" home. Empty nesters, especially as they age, have needs that are different than when they had a big family living at home and often will move to a house with wide hallways and doors and all on one level (or at least with a master bedroom downstairs) so that when the time comes that stairs are difficult for them, they don't have to leave their home. At that point, they can sell the big house, cash out a TON of equity, and buy their empty nester home (which yes, is sometimes remarkably similar to a starter home because once again they are a couple without children).
Real estate is generally a pretty big part of most people's portfolio. I guess it COULD be a matter of keeping up with the Joneses for some people, but financially it really makes a lot of sense to buy what you can afford to get into the market when you are young, upgrade when you can afford to, and cash out that equity when you no longer need the space. Financially it doesn't make a whole lot of sense for people that age to upgrade to an even BIGGER house, but at the same time I can see how people who have worked hard their whole lives and finally can afford the "dream house" would want to go for it.
I never understood why people are so judgemental of things other people choose to do with their money. It's almost like it gives people some perverse pleasure to sneer at people who make different financial decisions than they do. You don't always know the whole story and sometimes the simplest solution isn't the most financially savvy solution. Like anything else, a mortgage is a financial tool that can be VERY useful, or if used incorrectly, very dangerous. I try to not judge other people without the benefit of seeing their entire financial situation no matter what they drive, what kind of house they have or anything else that may be obvious to people on the outside. Sometimes that doesn't tell the whole story.