We once only wanted to have dessert at the Auberge de Cendrillion and the stuff there asked us to come back later during the day because it is not possible to have only dessert if other quests want to eat a full meal. And so we came back at 3pm and only had Dessert.