We don't have color-coded towles or anything, but my husband will occasionally use my towel after he showers. I don't really like it when he does that, but it is really no big deal.
I typically use one towel for the whole week, hanging it to dry between uses. I'm just wiping clean water off of my clean body, so I figure the towel doesn't actually get that dirty.