I second the pumpkin juice. I wish I had bought a bottle when I was there 2 weeks ago!
The wands are great, if a bit pricey. We didn't go to Olivander's, but at other shops you have a choice of Harry, Ron, Hermione, Ginny, Draco, Dumbledore, etc., etc.
There are some great mugs, especially the one with the owl on it. There's a black, sort of bland (IMO) mug that says "The Wizarding World of Harry Potter". I bought a great mug with a picture of Hogwarts on it and the words "The Wizarding World of Harry Potter" appear when hot liquid (I believe) is added to the mug (I haven't tried it yet).
My daughter bought a Gryffindor tie. There are also Gryffindor scarves. My husband bought a plastic Goblet of Fire to put on the shelf in our "movie room" (Yes, we're big NERDS!).
There are also great candies like Chocolate Frogs, jelly beans, peppermints, etc.