I plan on packing both. Usually shorts are good for the day time during this time of year, and then at night you may need long pants and a sweater.
Always good to have layers packed!
When I lived down there and had an AP we would laugh that we could spot the tourists because it was freezing but they were trudging away in shorts & a tank top. I would bring a variety of clothes the weather can be all over the place and isn't warm 24 hours a day like WDW.