I'd start with trying simple cooking oil (the oil in mayo and peanut butter are what make those work too). Let it sit a minute, then scrub with just a paper towel. It will take some work, but all of it will usually come off if you keep at it.
If that doesn't work, then try Goo Gone. You can usually find small bottles of it at DollarTree in their small automotive/tool section.