Early morning is better weather. Early Nov could potentially still be pretty hot in the afternoon ... But the Trek sometimes offers a significant discount for afternoon tours because they're less desirable in hotter weather. Not sure if early Nov would be considered hot enough to offer the discount, you could call the tour line and ask.
Fwiw, I've done it twice, once was very pleasant and once was boiling hot - August. I loved both of them!!