sounds like going in the purse really wasn't the issue but the taking without asking and the attitude about it when questioned. Sorry. It stinks. Good luck separating the purse from the real issue and accompanying attitude. I have two teens and one soon to be (although you can't tell that he isn't by his attitude), it's a struggle most days getting them to realize that the world does not revolve around just them. Sounds like you might need to get DH to help with the respect of boundaries. We still have this problem with the oldest, he will say he "borrowed" something. We then ask him if he asked, "no," well then it's not borrowing then is it? It takes a while. You can only hope eventually they will learn.
As for my purse, I keep chapstick, lipstick and library cards in it and a few other feminine things. If they want to go in it, they are more than welcome to. I tend to keep my money, drivers license and debit/credit card in my pocket, simply because I am always running and forget to grab the purse. I also have three boys, I think taking of the chapstick is my only real worry.
Maybe have the child work around the house to pay for replacing it and the shipping? Sometimes seeing what things cost in the way of work helps them realize that not everything is free and easily accessible. I regularly had this talk with the oldest about dual-layer dvd's, especially when they were fairly new and cost quite a bit to buy. Once he had to replace them and realized how much they were, what I had to do to get them (we ordered online, I'm picky about the brand & dye used), he finally realized that asking was a good thing to do. He isn't as picky as I am about those things, so I've also taken to buying him his own media, but he now asks before taking, so apparently something finally sunk in.