I agree with hit-or-miss, and that can vary from store-to-store. We have a fantastic produce section--really, really fresh stuff--while the next town over definitely doesn't. I coupon, so a most items I get for cheaper from other coupon stores, but produce and staples (milk, eggs, flour, etc.) don't really go on sale with coupons at other stores--and these are always consistently low-priced at Aldi, and that's what does it for me. I' always pleased with their tomatoes, mushrooms, potatoes, carrots, bananas, squash, and cucumbers ... also, they often have stellar sales on green grapes (and Disney brand for that matter ... who knew?). I also get their frozen stuffed pasta, and quite a few of their German products (including the chocolates kept up at the front

. Oh, and I love those frozen fruit bars that they carry.
One thing I can recommend to stay away from--Aldi's boxed macaroni and cheese. I like to keep that on-hand because my toddler's a macaroni nut, and Aldi's just doesn't taste good, IMO. I'll take Kraft/Great Value any day over Aldi.
If you're really looking to save money, I'd look into couponing (I use Southernsavers.com, spend about half-an-hour a week couponing, and save 40%-60% on my groceries pretty consistently--I'm sure there's a site that has stores local to you!) as well as using Aldi for produce--but if you're short on time and simply want to save here and there, especially on produce, Aldi might be the store for you.
ETA: My Aldi only accepts cash, debit and (I think) EBT. No checks or credit.