Hopefully it won't be too bad, if you're traveling in the off-season! But keep in mind that all resort guests get to enter the Wizarding World an hour before park opening. So, there'll already be people there by the time you are allowed in.
My advice would be to get to the gates at 8:30, because sometimes they start letting regular guests in early. Head straight to the Wizarding World and spend an hour or so there, then leave to explore other parts of Islands of Adventure when it starts getting crowded. Come back for lunch at the 3 Broomsticks at around 11am (before the lunch rush starts), and then leave again. Come back an hour before closing - the lines will have probably thinned out again by then.
Don't do Olivanders if the line is longer than a turn or two in queue. It's not worth a very long wait for a 5 minute show!
The Frog Choir and Spirit Rally shows are good choices for the middle of the day when it's very crowded. They often run every half hour, alternating - just ask a team member, and then sit down on the edge of the show area and wait. The show area is a semi circular piece of pavement in front of the Dragon Challenge coaster entrance. It has a couple of Stonehenge-like monoliths at the back. You can sit right at the edge of the pavement and get a great view.
If you're inside a shop (Derivish and Banges for clothing and souvenirs, Honeydukes for candy and toys) when the park closes, they won't throw you out right away. You'll have about 15 minutes to shop in relative peace.
Also, there's lots of Harry Potter souvenirs in the big stores at the front of the park. Don't forget to use your
AAA/CAA card!
Have a wonderful time! We loved our visit last December. My trip report is in my sig.