Loungewear. Holiday colors if you have them. Casual is the typical look for the day.
Years ago, back when we were local to both families and friends, we hosted a brunch. We stepped it up to every day clothes but festive colors for that.
Hopefully next year we can do a big brunch again. That's my favorite meal to prepare.