They're different things, in my vocabulary. If I'm going to the grocery store, I'm going to the little local place with kind of a crap selection. If I'm going to the supermarket, I'm going to one of the huge chain mega-stores.
I hear them used more or less interchangeably pretty often though.