Huge is relative. This particular weekend is Jersey Week along with the Food & Wine marathon, last week of Food & Wine festival, which all rolls into Veteran's Day which is a federal holiday. Crowds are higher than other times of November, but they are in the 5-6 range out of 10. We're not talking Spring Break crowds here, so I wouldn't let it deter you from traveling this time.