Well first, who said it was formal? If it were your nephew or any other guy, other than a very fashion-forward one, formal could mean no jeans.
I'd wear a comfortable sundress type dress.
Formal means long dresses & dark suits for the men...if not a tux. I would call your Nephew's mother & find out what they mean by formal.
If you are going to be on sand you can wear flats