The little things tend to add up, so if you can't find items on sale, I wouldn't bother.
Go with a theme--
soothing: little lovey blankie, pacifiers, cd of baby lullabies, night light, teething ring
play: toys for different stages (check Target and Kmart for toy clearance now)
bath: bath toys, wash cloths, bottle of baby wash, hooded towel
basics: burp cloths (pack of cloth diapers), onsies in different sizes, socks, bibs, flannel blankets
diapering: pack of diapers, diaper cream, vaseline, disposal bags for on the go, pack of wipes, "dipes and wipes" set like this:
http://amzn.com/B002WYJTPY
books: you can find lots of children's books in bargain bins at Barnes and Noble.
Another thing you can do is buy clothes of different sizes (lots of out of season stuff is on clearance at Babies R Us and Carter's-- search for Carter's store coupons online), buy a pack of clothes pin and a clothesline and "hang" the clothes. Place this in an inexpensive laundry basket (woven willow laundry baskets at Michaels' are just $4 or so with a 40% off coupon). You'll get a lot of ohhs and ahhs over the presentation. Even if you buy a pack of onesies, socks and just 2 or 3 outfits, once everything is separated on the clothesline, it'll look like a lot! You can even buy a two pack of one-piece play clothes from Target (Circo or Gerber brand) for under $10. Add some bibs too!
If you have an Ikea near by, they have lots of reasonably priced children's items. They have bibs, towels, step stools, nice wooden toys, etc all at great prices.
If you're crafty, make some personalized onesies with iron-ons that you create and print.
Don't feel like you HAVE to go with registry items. You're giving a gift and that's what counts.