I'd actually love to hear myself from a PhotoShop pro who has used PaintShopPro and could tell us the differences. I am sure there are a number, otherwise it could never justify the cost difference.
Since I am pretty familiar though with PaintShopPro, I can talk to that. I have not upgraded to version 10, so I am still on PSP 9. I have used it since version 4 or so for web graphics, icon creation, photo editing, etc. It has many of the same features that PhotoShop does. My sense is that it trails PhotoShop by a version or so in many ways - features, reading PhotoShop files, handling PhotoShop plugins. If it can't handle the newest PhotoShop plugins, often the vendor has a PSP specific plugin. Again, though, most of these are out of my price-range, and I find a way to do what I want to do within the bounds of the product usually, and it's amazing what you can do with it.
If you can get it for under $40, I personally think it's a deal! It aims to be like PhotoShop, and it's close enough for most people without hurting the wallet quite so badly. I don't know much about Photoshop Elements and how stripped down it is from Photoshop itself. If it's a major step-down, I would imagine PSP would be better (and it sounds like it's less expensive as well.)