We bought a house with less than 20% down. Fortunately, housing prices were still heading up, so we were able to get rid of PMI ahead of schedule. And PMI is the primary extra cost to buying with less than 20%.
I would worry more about your monthly payments and your ability to cope with whatever maintenance issues are likely. There are lots of good reasons for buying with less than 20% down. But real estate agents often encourage people to buy at the largest mortgage they can qualify for, and that's so often a bad idea. We were hi-tech DINKs with great job security but new to the job market, and could have qualified for a house at least $50K more than what we bought, but knew better. Even today, with mortgage lending getting tighter, they're still willing to lend you money based on a household budget that will keep you fed, clothed, with retirement savings and an emergency buffer to cover anything short of long term job loss - but not with enough money left over for regular WDW trips.
So as long as you're comfortable that after your monthly payments (including PMI, taxes, and insurance) and other fixed expenses, you still have enough left over to contribute the max to your retirement plans, save for any college needs, and build a six-month emergency buffer, without living off of ramen, don't sweat the PMI. Do pay attention to the rules, and either refinance or get the PMI taken off when you qualify. In our case, all we needed was an appraisal showing the house had gone up in value to make our equity clearly more than 20%.
EDIT: The other big downside is being upside down when the market drops and catastrophe strikes. Many of the people who lost their homes during the recent bust had put down less than 20%. I don't think we'll see another bust like that for a while, but it can happen. Make sure you have long term disability insurance, and be realistic about job security.
