Some of that may depend on where you're from, too. What makes a Southern cold is still balmy for a Northerner!
We've gone in the first week of November each of out three trips. I usually wear light-weight long pants, but my wife wears shorts. We've been comfortable with just a light jacket, and that is almost always stashed away in the backpack or locker by lunch.