class output cache: aren't there too many files in a single directory?

[eluser]Bad Penguin[/eluser]
I've enabled the output's cache on a website with like 69.000 pages
all the cache's files are created in a single directory
isn't this a performance problem when there are too many files in a single directory?

does it make any sense to extend the output class to use a 2-level subdirectories
system for those cached views?

[eluser]atno[/eluser]
[quote author="Bad Penguin" date="1306783853"]I've enabled the output's cache on a website with like 69.000 pages
all the cache's files are created in a single directory
isn't this a performance problem when there are too many files in a single directory?

does it make any sense to extend the output class to use a 2-level subdirectories
system for those cached views?[/quote]

on unix systems with ext3 each directory can have up to 32k files/folders, yes you better split your cache files in multiple dirs within your cache folder

[eluser]WanWizard[/eluser]
Depends a bit on the filesystem used, and the speed of the storage system.

On Windows, you definately want to limit the number of files in a directory to a few thousand. On Linux (ext3), thousands of files is not a problem. You can increase the speed on ext3 by formatting the volume with the 'dir_index' parameter, which implements a btree algorithm for directory indexes.

How big are these cache files? Maybe you can get a better speed gain if you cache in memory (a simple memdisk would be enough, other solutions require a extended cache library).

[eluser]WanWizard[/eluser]
[quote author="atno" date="1306789707"]on unix systems with ext3 each directory can have up to 32k files/folders, yes you better split your cache files in multiple dirs within your cache folder[/quote]
Sub directories, yes, this due to the limit of links per inode. ext3 doesn't have a file limit.

I've had a directory with over a million files once, delay wasn't really noticable (with dir_index enabled). Most annoying was that most commandline tools stop working (forget doing an 'ls' in such a directory).

[eluser]Bad Penguin[/eluser]
[quote author="atno" date="1306789707"]yes you better split your cache files in multiple dirs within your cache folder[/quote]

They are not my files but the codeigniter's default cache system from the Output class