I used to be one who would consider something dirty if I wore it -- regardless of the amount of time. Threw on some jeans to walk outside and get the mail? Those go in the hamper. Put on a sweatshirt over my T-shirt b/c I was cold while I watched a couple episodes of NCIS? Dirty clothes!
A semester in Italy changed all that. First -- doing laundry was freaking expensive! Second -- I discovered that it was completely normal among my friends to wear something (particularly jeans) more than once before washing them (barring stains, smells, etc.).
Now, I vacillate. Sometimes I'll wear an item a couple times before washing, and other times if it touches my skin I feel like I need to toss it in the hamper.
And re: the towel thing -- I, too, will often use a towel twice (esp. b/c I use one for hair and one for body) before washing it. But I remember a Dilbert comic where the characters talked about that, and one said, "I'm the cleanest thing in the house when I get out of the shower, so the towel actually gets cleaner each time I use it." Then in the last frame, he says, "Are towels supposed to bend?"
Hee!