Another vote for Penzey's for the important ones. They are not as pricey as you might think, because they sell in jars and bags, and the bags are much cheaper if you don't need a jar. I double-box the bags inside two tupperware bins in my freezer, to keep them cold AND dry between jar refills.
For the basic things that I need a lot of, I buy at Asian markets. They tend to have large bags of things like pepper and granulated garlic at reasonable prices. (Cinnamon is a Penzey's thing for us, though -- we prefer Ceylon and Vietnamese, and the easiest place to find them for me is Penzey's.)
I would not buy dried herbs at a place like Dollar Tree. Those are old as the hills, and you easily have to use 3X as much to get the same flavor.