I find a lot of good deals at Tuesday Morning, which is a chain that sells stock after it has been discontinued from other retailers. They seem to get stock from Target and Michaels. They often have paper packs, 25 sheets for $1.99, or some brands of paper stacks for $3.99. They also always have albums.
But the key to controlling scrapbook costs is only buy what you truly need, and don't get sucked into trends and having the latest and greatest. My basics list:
Album & page protectors
Paper Trimmer
Good pair of fine tipped scissors
Metal ruler
Black Journaling Pen
Adhesive
Cardstock and Patterned paper to match your current project.
Everything else: stickers, brads, eyelets, ribbon, buttons, chipboard, fancy alphas, flowers, etc. They aren't really necessary. And if you only pick up some of these things occasionally, and have an idea on how they will be used, it will help keep your budget in check. And like everyone says, shop sales and coupons.
Now, if you've decided that scrapbooking is your thing, and you aren't going to get tired of it and move on to a different hobby in a few weeks, there are 3 "big ticket" purchases that I recommend. I think these will save money in the long run.
Making Memories Paper Trimmer. It retails for $50, but you can buy it with a 50% off coupon at Michaels. It has a self-sharpening blade. With the basic Fiskars type paper trimmers, you end up spending quite a bit of money on blades.
ATG tape Gun - I ordered mine from
http://www.framingsupplies.com/3M/ScrapbookersTapeGunsandTapesPage.htm , although Michaels is just starting to carry the pink version. The cost for the gun with shipping is over $50, but the tape refills are about $3.50. The stuff you buy in the scrapping department can cost $5+, and only last you a fraction of the time. Look at a tape runner, and the length will be measured in feet, this tape is measured in yards. And the big thing, it stops working. You might go back to your albums and see pieces falling off. Not with the ATG tape.
Cricut - Because buying letter stickers is expensive, and cutting them by hand gets time consuming. There are always good deals online, and with Black Friday coming up, there will be good deals in the stores.