It's just fine. It's not like the dessert at places are cheap. We went for just dessert at Kona Cafe and our bill was the same as what it would have been for a full lunch. You figure a lot of people don't order dessert at all. Now I would say it would be bad form to linger 2 or 3 hours over just dessert, but most restaurants don't require you to order appetizers and desserts with your entrees, so it isn't a revenue loss for them if you only order these items.
I do make sure I tip approrpiately to the time I am at a table though when I go for just appetizers or desserts.