We came in just after Labor day this year (and it was pretty busy) but last year we were there on Labor day and it was incredibly busy. The Friday was manageable but everyday after ended up being busier. On the Monday I couldn't deal with the massive crowds anymore and went back to the hotel.
I guess how busy it is is all subjective, but I really, really do not enjoy the parks when it's really busy (the labor day I was there everything was at least a 30-40 min wait and you practically had to walk single file through big portions of the park), that on top of the crazy heat was not enjoyable for me at all. I don't know how people do it over Christmas with those crowds.
I'm definitely not going back at Labor day again though lol.