I lived nearly ten years in the French Quarter. Here's a few personal must-dos whenever we go back:
Take a cemetery tour. I highly recommend St. Louis #1, located on the edge of the French Quarter, and Lafayette Cemetery, located in the Garden District. But they're all beautiful and historic. There is occasionally some crime, so sign up for a tour rather than exploring on your own.
Visit Bourbon Street to say that you've done it, maybe stop at Pat O'Brien's for a Hurricane or the Cat's Meow for karaoke contests, but don't spend all your time there.
Go down Lower Decatur Street to see where the locals hang out. It's the last street before the river. When you're facing the river, turn left. Lots of corner dive bars with cheap drink specials, and an awesome hole in the wall restaurant called Coop's. Plan to eat there at least once.
Stop by Jackson Square. Get a tarot reading, watch the street musicians or talk to the visual artists. There's nothing like it anywhere, especially on weekend evenings. Tour the St. Louis Cathedral while you're down there.
Take a ghost tour from Haunted History. There are several ghost tour companies in town, but Haunted History is the best/most authentic.
Walk through the really expensive antique shops on Royal Street, then head over to the Lower Quarter for the real bargains in the junk shops along Barracks and Basin.
Visit the French Market on a weekend afternoon.
Pick up carryout food from Quartermaster. It's a little convenience store with a fabulous deli counter in the back.
Go to Cafe du Monde for beignets and coffee. Have a powdered sugar fight while you're there.
Take the free passenger ferry across the river to Algiers and go see Mardi Gras World.
Take a dinner cruise on the Natchez or Creole Queen.
If you have a car, drive through the Katrina damaged area. Just ask a local for directions or better yet, have someone who was there for the storm go with you. It's heartbreaking, and sobering to realize how many things are still not rebuilt.
Buy a bottle of Vampire wine from Sidney's. It's actually a California wine, but sort of a New Orleans symbol.
Those are just a few suggestions off the top of my head. New Orleans is the sort of place that wandering out of your hotel can be cool. You never know when a second line parade will go by! Don't try to do a commando trip. Just relax and slow down, and let the experiences find you