Nope. Taken by just towels- used or unused- the chair is fair game. I have zero qualms about throwing the towels in used towels and taking the loungers. Anyone comes up to me while I’m clearing them out I’ll just say sorry and leave- but no one has.
I did one time at Sapphire Falls use loungers out in front of where I was seated one morning. They had stuff and towels but it had been half an hour with no one around. About the time I was leaving the party came up and was like those are our loungers. “Ohh I’ve been here 3 hours and didn’t see anyone” and grabbed my towels and went back to “my” lounger which conveniently now in the shade. I’ll be honest- I was sorta looking forward to the looks on their faces on that one. I had spent the day at the pool while they had reserved a prime pool spot, spent the day at the park and rolled in around 3pm expecting their spots. I don’t want a fight so of course I gave it up but that behavior is only getting more common.
Disney does have signs “no reserving loungers with towels and or personal items” but they obviously don’t enforce. Heck they rarely even go through collecting used towels that people don’t return.