Cheapest places are the clearance stores like Ross Dress For Less and T J Maxx. Limited ranges - they've got what they've got and when it's gone it's gone, but prices are usually half (or even less) those in the high street here.
The outlet stores can be good too e.g. the Calvin Klein store at Beltz on I Drive often has very good offers but obviously only stocks CK perfumes. Similar thing at the Polo Ralph Lauren store.
Best price/range combination we found last trip was at Eckerd drug store (there's one on almost every corner) but they had a special promotion on 'designer' fragrances at that time.
There are lots of options. Enjoy shopping around!