Page caching and maximum number of files per directory

Rails
1

I am using ubuntu with ext3 file system.

How many files can I store inside a directory?

Is it an effect way to save all 100,000 cached files inside one
directory if it is allowed? Or will it be worth arranging these files
into sub directories to reduce the file size, to speed up web page
request?

2

Nanyang Z. wrote:

I am using ubuntu with ext3 file system.

How many files can I store inside a directory?

Is it an effect way to save all 100,000 cached files inside one
directory if it is allowed? Or will it be worth arranging these files
into sub directories to reduce the file size, to speed up web page
request?

I cache fragments with an explicit path for the type of fragment cached
and model.
Has made it easy to clear out some fragments while keeping other if I
change the rendering of a certain piece. Like:

/show/project/ <- individual project fragments go here
/show/scenario/ <- scenario fragments go here
/related/project/ <- “relations” fragments for projects go here
/related/scenario/ <- “relations” fragments for scenario go here

along with keeping the file counts per folder lower, if I change the
“show” rendering of a model, or all models, I have a specific tree to
prune to dump those cached fragments. Same goes for the “related” tree
(which is comparatively expensive to generate versus the “show”
fragment).

3

Ar Chron wrote:

I cache fragments with an explicit path for the type of fragment cached
and model.
Has made it easy to clear out some fragments while keeping other if I
change the rendering of a certain piece.

I think it is a good solution for you.

In my case, I want to cache the whole page, and there will be more than
100,000 page, which sharing a common layout. I want to know if I should
put all the cached files inside one single directory. By default, rails
will do so. But I am worry it may go beyond the limit of the file system
to store so many files, and to put too many files inside one directory
may make fetching one file take more time.

4

Hiya,

100,000 files in a directory sounds a lot.

In ext3 I think the number of files is irrelevant to directories. Its a
matter of the drive itself, size of inodes and size of all the files,
etc.

I seem to remember seeing a presentation from Joyent where they found
they need a limit of 32,000 files in a dir, from a practical point of
view - could be wrong though I can’t find it at the moment.

ext3 - Wikipedia for more info.

If its not to much effort, it would be safer to split them into smaller
chunks.

Matt S.

5

Matt S.1 wrote:

In ext3 I think the number of files is irrelevant to directories.

So, I think the directory on my server can hold 100,000 per directory.

If its not to much effort, it would be safer to split them into smaller
chunks.

These cached files are named by article titles, so it won’t be too easy,
and that is why I want to know if it worth the effort.

BTW, in your opinion, how many will be a nice “smaller” size?