We are visiting exactly the same time - from past experience hoping it's going to be pretty quiet (ish) - although we haven't been since Harry Potter opened but have been the last 2 weeks in August a couple of times before.
We are spending the start of our vacation at Disney - it's free dining season - and as such I reckon the Disney parks will be busier but now unmanageable as a lot of schools are back in session. If you are looking for TS dining though, I would book asap as the restaurants will be very busy - I tried to change times on a couple of ADR's, unsuccessfully as they were already fully booked.
P.S. Just wanted to say as well, since you're clearly a HP fan - we are going on a Harry Potter walking tour in London next month - visiting various sites from the movies