I just wore a pair of swiming shorts and tshirt which was fine for eating in the buffet restaurant. If your planning on having lunch in one of the table served ones i would dress up to smart casual for that.
I didn't really see anyone wearing swimwear unless they were in/near the pools (obviously!). Most people I saw looked fairly casual and wore what they would probably wear around the parks. Personally, I tend to wear cotton dresses which are cool and comfortable but also look smart enough to eat in a sit-down restaurant. Having said that, I still get changed for dinner!
Shorts and T for me unless I was specifically going to the pool deck for a swim or grabbing some sun then obviously would wear my swimming costume.
A Saress is brilliant if you're in your cossie but want to grab some food or something in the buffet (I'd not wear to the table restaurants though).
Also, shorts and T's are absolutely fine during the day in the table service restaurants, you can think of them in the same way as you do the park restaurants - with the exception of Palo Brunch.