As you no doubt have found, these folders are fairly awful to store in any meaningful sense and I'm reasonably confident that they weren't actually intended for long term storage, or to be stood up, unless in a specifically designed circular cabinet (I've seen one in an office supplies shop and it was kinda neat- dimensioned perfectly to take a bunch of identical folders with a max number of pages and rotating to give access without walking round the cabinet. )
You have, however, said that you don't want to obtain new folders or specific cabinetry which limits your options (effectively, you're asking the impossible, trying to find a solution to a problem caused by this mishmash of folders being a crap solution to the problem of document storage)
In an ideal world I would recommend to standardise your folders, or replace them with box files (all round superior for storage, less so for using the contents like a book), or replace them with a concertina file, or replace them with hanging folders in a filing cabinet
I've had a proliferation of these folders before and found storing them so their spines are pointing to the ceiling and the front/back covers are vertical kept the contents in better condition - they're hence stored slightly open and thus are more cuboid shaped than wedged shaped. They get support from each other but need a book end. Works best when the inner clips have four rings rather than two. Also doesn't work well for weak ones like the plastic
Alternatively some device that has fingers sticking out along a strip that is secured to the wall, with the folders being supported in their current orientation by the fingers- effectively each folder having its own pigeonhole/slot in the supporting device. This needn't be a complex device; even a length of foam based central heating pipe lagging with a slot for each of the folder covers to wedge into..
All in, my vote - toss them out and get a filing cabinet. Or scan/photograph the contents so you can refer to an on screen copy of the document, then store the paper versions in their files, flat as someone else advised (stack alternately so they stay level as the pile rises)