Contact the buyer, let them know the item arrived, and while you are disappointed in the quality of the fabric, you are most disappointed in that they are not true to size and do not fit. If necessary, offer to send a photo of the scrubs they sent next to some other scrubs in the same stated size to illustrate the difference.
Even though they state "no refunds or exchanges," a refund and exchange is definitely warranted. It's also possible that if they had other scrubs in different sizes for sale, they may have mixed them up and shipped the wrong ones. If they do refund or exchange, most likely they won't do so until after they've received the scrubs back. Be sure to purchase some sort of confirmation -- delivery confirmation, signature confirmation, insurance, etc.
If they refuse to refund or exchange in the first response, reiterate the disappointment in the size error and try to say something about the item being misrepresented. Even if they were going by what they were told about the size, it's still their responsibility to verify the size.
If they absolutely refuse to refund your money or exchange the item, be prepared to leave negative feedback. DON'T say anything about feedback when talking to the buyer about a refund/exchange; that could be construed as feedback extortion and is against eBay policy. Besides, I'm sure they'll get the idea that negative feedback may be forthcoming. Stay factual in the feedback commentary, don't get personal or insulting in any way. Be ready to receive a negative in return, but if your feedback score is high a negative is going to hurt your seller a lot more than it will you. You can also leave a response to negative feedback you receive, so keep that in mind if they do neg you.