Overclocking is like turbocharging an engine: more performance, more heat, less reliability, and shorter life. There are valid reasons to overclock but for most purposes it is neither necessary nor desirable.
I
might disagree with this analogy depending on how you meant it. A quality factory-designed turbocharger does not create less reliability or shorter life for the engine. (Just ask your average over-the-road trucker who's probably got a turbo diesel under the hood.) Adding one yourself will probably be less reliable and cause shorter life, but again, if done right (lowering compression ratio, etc), it can be done fairly safely.
Also, there are times when overclocking is unlikely to damage your processor. Many times, various-speed processors are identical, it's just how good the yield is that determines what they're rated for. If all the CPUs in a single big wafer (I think I'm using the right term there, but I might not have the same right) pass the tests, some will still be marked as slower because they're can't sell just the fastest ones.

They are then set to run at a default clock and bus speed - maybe locked, maybe not.
One great example of overlocking were Intel's 300mHz Celeron CPUs - cheap and most could overclock to 450mHz perfectly. (By increasing the bus from 66mHz to 100mHz.) A 50% overclock and rock solid - I put together several of these and the one that I used most myself was maybe the most reliable computer I've ever used.
Anewman said:
What I find funny is that persons sometimes spend much more on cooling parts then it would have cost to simply upgrade the cpu, so why not just upgrade and save the headache?
Well, depends on what level you're looking at - the cost of top CPUs ramps up quickly, Intel has been selling "extreme" CPUs for a few years that are nearly $1,000. If you can equal that speed with a $300 processor and $100 of cooling parts, you're ahead of the game. But you'll still be slower than the processors available 6-12 months later.

There's also appeal of not only tinkering in general, but getting more than you paid for.
One other thing to note is that overclocking is a fairly accepted process nowadays. Most motherboard manufacturers make sure that their motherboard has flexible overclocking options. Some companies sell graphics cards overclocked past what they're rated for by the original maker (usually ATI or nVidia.) Standard video card drivers have overclocking built-in - my ATI gaming card has a standard "overdrive" feature in the driver that will automatically overclock the card as far as it can go while still being stable.
Now, to respond to Jann's original question - no, you don't need to worry about it whatsoever; your average PC user will never overclock their system. For speed, you'd have a hard time finding a 3gHz processor now, most are slower but do more per cycle and have two CPUs in one, hence the "dual core" feature. Just adding mega/gigahertz to speed up the PC was a dead end and Intel took a loooong time to realize that!