Most of our trips have been the week before and after Labor Day. The week after Labor Day is significantly less crowded than the week before Labor Day. While I understand from other posters that compared to other holidays, Labor Day weekend is slow, my experience is that it IS NOT SLOW when compared to the days before and after that weekend. There is a noticeable crowd difference on that weekend compared to the days before and after. For example, I waited 50 minutes for the Nemo ride in Epcot on Sunday of Labor weekend at 11am last year, and no, there wasn't a ride malfunction.
Since you are arriving on the weekend, be prepared for the a moderate crowd on the weekend, but the rest of the week is quite pleasant as far as crowds are concerned.
Also, be prepared for rain. Our 2015 trip was almost rain free. But our 2016 was rain filled -- not just pop-up thunderstorms, but rainy days, hours of rain. The rain chases the crowds away though, so it can be a good thing.