Going to a restaurant for dessert and/or drinks IS a thing. There is nothing unusual about it. A normal party having just dessert would most likely tie up a table for a fraction of the time compared to eating a full meal.
Go, have dessert, (I believe many Disney Signature restaurants do a great job with desserts), maybe have a drink. Tip on the amount of your bill, there's no need to tip on hypothetical entree's that you didn't eat.
Even though going to a restaurant for dessert is a normal behavior, you will never see a restaurant where 50% or 75% of the tables are having dessert only.
I have stopped for drinks and dessert and would bet our tab was higher than some people eating dinner at the same restaurant.
In the course of an evening the difference in a servers income will not even be noticeable if one of their tables was for dessert only.
It is not rude, for heavens sake, don't even give it a second thought.