You should be fine. The
HHN crowd won't start coming in until 3-ish, and that's on the Studios side, where the event is being held. They like to get a few rides in before staying over for the event.
I do recommend avoiding Saturday and Monday. No HHN on Monday, but the parks will be crowded because it's a holiday.