The most important thing about keeping your data is redundancy. If you're not running out of room on the drive in your computer, don't delete them. You don't really gain anything by keeping a large amount of free space on your hard drive.
My usual technique is burn two DVDs; one stays in a big CD wallet and the other goes in my fire safe. I've recently been converting all my old CDs (including 10+ year old ones) and had extremely few fail due to age; I think that fears of all your CDs and DVDs becoming unreadable after a couple years are overblown. As long as my discs last long enough for the next disc technology, I'm happy.

I'm sure I'll be redoing my DVDs onto BluRay discs in the next few years.
I use two external hard drives (one for backing up everything, including pics and music, and one for backing up just the pics and purchased music up again. They are never plugged in unless I am backing up.
Also all edited pics are uploaded to Smugmug, so I consider that extra backup.
I used to use CD's but external hard drives are quicker and can hold so much more. And they cheaper in the long run.
Hard drives are absolutely, positively the least reliable storage method. Even if they just sit on the shelf. And external drives can be especially troublesome as it's more difficult to notice when they're going bad, and most hard drive scanning tools don't like running on a USB drive. Having multiples alleviates much of this but ultimately, your hard drive is going to be the first thing to fail. There's a reason why hard drive manufacturers have mostly gone to one-year warranties instead of the three-year that used to be the norm. (Seagate is the primary exception, with five-year warranties across the board.)
As for cheaper... well, DVDs can hold around 4.35 gigs for maybe 20-25 cents... that's about 4.6 cents per gig. The best bargain in hard drives is a 500g for usually $110 or so, more for an external, that's about 22 cents a gig. You can burn four DVDs and it'll still be cheaper than hard drives.

And in the long run, even if you have something insane like 5% failure rates (you're probably looking at less than 1%, really), you still have the vast majority usable, but when your hard drive fails, it's useless unless it still has a warranty on it.