I have just booked for Christmas 2016, arriving 22nd for 2 weeks. Like you we have only been in April, May and June. The weather here is regularly below freezing so I will take anything above that. I have been watching hundreds of vids to look at crowds etc and I think if we know in our head it's going to be busier, we won't be too stressed about it. We know what we like now and what we would avoid so we can soak up the atmosphere and appreciate all the beautiful decorations.
It looks in the vids most people in shorts and t shirts with a sweater on in the evening.
Also only saw 1 when the crowds were massive and that was just prior to wishes kicking off!
I just approached it like this, if we were home we would have a beautiful tree that I could enjoy after work, after cooking, after cleaning and while the heating wason full blast daydreaming about when we could be back at Disney,or the most magical time of the year could be enjoyed in the most magical place on earth. I'm on team DO IT!!