When you look through the lens, you cover the viewfinder with your eye. When your eye is not there, light is free to come in through the viewfinder, bounce off the mirror, and hit your exposure meter. This can cause you to underexpose because the camera thinks the scene is lighter than it really is.
Once you have set your exposure, you don't need a viewfinder cover. While the picture is being taken, the mirror is out of the way and no light should be able to enter through the viewfinder.
Use a viewfinder cover when you are taking pictures in a non-manual exposure mode and you are not covering the viewfinder with your eye (or anything else). Typically, cameras have a viewfinder cover that attaches to your camera strap, so it's easy to put on. Some cameras come with a built in cover that you use with a little lever near the viewfinder.
This is only an issue for SLRs and DSLRs, not point & shoots or rangefinders.