We visit St. Augustine 2-3 times each year, &, in fact, just returned from a 2-week visit this past Wednesday.
You can easily do the fort in 2-3 hours.
November should be pleasant weather-wise - in summer, the fort can get miserably hot & humid, especially if you’re not used to heat in the south.
During the holiday season, St. Augustine has “Nights of Lights,“ &, if I’m remembering correctly the lights may be officially turned on either the week before Thanksgiving or the weekend directly after.
”Nights of Lights” is extremely popular, & the crowds will continue to grow the closer it gets to Christmas.
The historical district - where the fort is located - is also small & doesn’t always handle the extra crowds & traffic well - traffic can get very congested. The earlier in the day will be less crowded - the crowds build throughout the day.
That said, all the lights are gorgeous, & the historical district is just beautiful during the holidays.
Across from the fort is the Pirate Museum which was designed by a former Disney imagineer & is very well done.
Our favorite restaurants in the historical district are The Columbia, Floridian, & Harry’s. OC White’s is good too. Both Harry’s & OC White’s have ghost stories associated w/ their buildings. Our favorite brunch is The Ice Plant - which is located over the St. Augustine Distillery. We also really enjoy Prohibition Kitchen. Casa de Reina & River & Fort are 2 relatively new restaurants, & both have good food & gorgeous ambience.
The Tini Martini Bar is a great bar w/ a lot of history - they famously stayed open during one of the hurricanes & were featured on CNN.
The fort is right there at the Bay which is a lovely place to walk, &, across from the fort, is the historic district w/ tons of interesting shops & little cafes & bakeries in addition to the restaurants.
EDITED TO ADD - the Fort is NOT one giant junk store. The architecture of it - made out of coquina - is amazing, & there is so much history & culture to the fort. Yes, it’s older & is showing its age, but it’s a wonderful piece of not only American history - but also world history.