[SATLUG] ext3 file number limitation

John Pappas j at jvpappas.net
Fri Jul 18 17:07:57 CDT 2008


On Fri, Jul 18, 2008 at 11:40, Channing <Channing.ML at channingc.com> wrote:

> Sean I wrote:
>
>> Is there a file limitation anyone knows of in a single directory on
>> ext3.  We are creating a half a million openssl certs and after we get
>> to about 200,000 in a directory and are having problems with slowdown
>> of openssl.  We do have plenty of inodes, but i have never tried to
>> create this many files before.
>>
>> Sean
>>
>>
> I don't know of a hard limit, but I've heard and practiced the philosophy
> of limiting to around 5k in a single directory.


I am not sure that using a different file system is feasible in this
application, but XFS handles large file trees better than ext3.

But let's focus on Ext3.  Referencing <
http://wiki.archlinux.org/index.php/Ext3_Filesystem_Tips>, you could see if
enabling "Directory Indexing" on the effected filesystem helps (Check if it
is enabled in the first place):

mine:~# tune2fs -l /dev/sda1 | grep features
Filesystem features: has_journal resize_inode *dir_index* filetype
needs_recovery sparse_super large_file

To enable if it is not:
`tune2fs -O dir_index /dev/[hs]dXY` to enable the indexing
`e2fsck -D -f /dev/[sh]dXY` to re-index existing directories.  IIRC the file
system should be unmounted, but I do not think that it has to be.

Also, the old `noatime` mount option could help.

Last shot in dark suggestion, enable reservations/preallocations
# mount -o reservation

Let us know if any of those help!
jp


More information about the SATLUG mailing list