I struggled with this problem myself a few years ago.
I now put them all into five sections:
1. Studio recordings
in year order. I include all alternates in this section so for example Essential elvis cds would be close to the years that the cds contain.
2. Live recordings
. Again done by year. I include all the audience/ soundboard recordings as well as official cds like aloha.
3. FTD cds
. This time in order of release.
4. Import releases
from japan, argentina, USA etc
5. New compilation
releases such as Elv1s and E2. In this section i include all compilations such as greatest live and greatest love songs.
I also number each cd ( as Colin does ) and keep records on excel. I don't locate them on the shelf in number order, its just to keep an eye on the number of cd's i have.
This system works for me well. Once you remember where each section starts and ends you can find the cd you require easily.