[SATLUG] ext3 file number limitation
j at jvpappas.net
Fri Jul 18 17:07:57 CDT 2008
On Fri, Jul 18, 2008 at 11:40, Channing <Channing.ML at channingc.com> wrote:
> Sean I wrote:
>> Is there a file limitation anyone knows of in a single directory on
>> ext3. We are creating a half a million openssl certs and after we get
>> to about 200,000 in a directory and are having problems with slowdown
>> of openssl. We do have plenty of inodes, but i have never tried to
>> create this many files before.
> I don't know of a hard limit, but I've heard and practiced the philosophy
> of limiting to around 5k in a single directory.
I am not sure that using a different file system is feasible in this
application, but XFS handles large file trees better than ext3.
But let's focus on Ext3. Referencing <
http://wiki.archlinux.org/index.php/Ext3_Filesystem_Tips>, you could see if
enabling "Directory Indexing" on the effected filesystem helps (Check if it
is enabled in the first place):
mine:~# tune2fs -l /dev/sda1 | grep features
Filesystem features: has_journal resize_inode *dir_index* filetype
needs_recovery sparse_super large_file
To enable if it is not:
`tune2fs -O dir_index /dev/[hs]dXY` to enable the indexing
`e2fsck -D -f /dev/[sh]dXY` to re-index existing directories. IIRC the file
system should be unmounted, but I do not think that it has to be.
Also, the old `noatime` mount option could help.
Last shot in dark suggestion, enable reservations/preallocations
# mount -o reservation
Let us know if any of those help!
More information about the SATLUG