The truth is that, if your clothes are not clean enough to go back in your wardrobe, they really need to go in the wash, not on your chair for wearing again.
If you've warn them and they're even slightly smelly or dirty, then you really don't want to wear them again, do you? So, wash them!
On the other hand, if they're neither of those things, they won't contaminate other things if they go back in with the clean stuff, so do that.
Similarly, if they're too crumpled to go back with the tidy stuff, they're too crumpled to wear, so press them before putting them back.
Now, you might argue that some clothes will never get washed following this principle. In principle, this could be true, but if your clothing is clean enough to go away again, I don't see the problem. And in practice, this simply doesn't happen. You can tell when clothes need washing.
I made this shift myself, about 2 years ago, and can attest that it works perectly well.
In summary, then, I'd say your challenge isn't to find a different resting place for half-worn clothes, but to see the problem differently.