Orchid XP v8 wrote:
> No. It was a first course in filesystems. I imagine the guy picked est2 
> because it was easy to look up the reference material. 

Well, he still failed. Or you misunderstood what he was saying. :-)

If you want to know how ext2 works, look up how Unix v7's file system worked 
about 30 years ago. It's essentially the same, except in *where* it stores 
things physically on the disk. The concepts are all the same.

> While we're on the subject... NTFS has an optimisation where "small" 
> files are stored in the same block as the directory entry. (Saves 
> seeking and wasting half a disk block.) Does est2 have any optimisations 
> for small files?

NTFS's "i-nodes" (called MFT records) are some 1K to 4K in size. Ext2's 
inodes are closer to 64 bytes or something. There's no slack space to speak 
of in an ext2 i-node.

Anyway, the idea of the data being stored in the same place as the 
attributes on NTFS is based on the fact that the data, the permissions, and 
the locations where other data is stored, is all the same sort of "stuff". 
If your list of permissions gets too big, or your file has lots of 
fragments, those too might wind up being stored in the "data" area of the disk.

-- 
   Darren New, San Diego CA, USA (PST)
   I ordered stamps from Zazzle that read "Place Stamp Here".

Post a reply to this message

TC <do-not-reply@i-do get-enough-spam-already-2498.com> wrote:
> To all Linux gurus here: can anybody tell me how many files can be stored in 
> a Linux directory without performance degradation? Or is there no limit for 
> directory entries on Linux file systems?

http://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits

  I think most file systems (such as ReiserFS) have O(log n) indexing for
files in a directory. So they don't get degraded at all.

-- 
                                                          - Warp

Post a reply to this message

Warp wrote:
>   I think most file systems (such as ReiserFS) have O(log n) indexing for
> files in a directory. So they don't get degraded at all.

It's an option called "dir_index" on ext2/3 that you can set with tune2fs or 
with the appropriate fsck (and of course during mkfs).  Just in case you 
ever need to know. :-)

-- 
   Darren New, San Diego CA, USA (PST)
   I ordered stamps from Zazzle that read "Place Stamp Here".

Post a reply to this message

>> No. It was a first course in filesystems. I imagine the guy picked 
>> est2 because it was easy to look up the reference material. 
> 
> Well, he still failed. Or you misunderstood what he was saying. :-)

It was... my God... about ten years ago now. o_O

>> While we're on the subject... NTFS has an optimisation where "small" 
>> files are stored in the same block as the directory entry. (Saves 
>> seeking and wasting half a disk block.) Does est2 have any 
>> optimisations for small files?
> 
> NTFS's "i-nodes" (called MFT records) are some 1K to 4K in size. Ext2's 
> inodes are closer to 64 bytes or something. There's no slack space to 
> speak of in an ext2 i-node.

Sure. I was just wondering if ext2 does anything special with small 
files, that's all.

Since files can only be allocated an integral number of data blocks, 
really tiny files potentially waste an entire block. A directory full of 
millions of tiny files could actually eat quite a lot of space. But by 
putting that data inside the directory itself, you avoid all that wasted 
space, and save on some disk seek time to boot. It seems like a neat trick.

Then again, I've sometimes wondered what would happen if you had some 
filesystem that split the disk into several seperate regions with 
different block sizes, and allocated files accordingly. (I.e., put the 
really huge files in the area with big blocks, and the tiny files in 
some area with tiny block sizes.) I rather suspect you'd permanently be 
running out of whichever size you happen to need the most tho...

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Orchid XP v8 wrote:
> Sure. I was just wondering if ext2 does anything special with small 
> files, that's all.

Right. It could, for example, pack data from multiple small files into one 
sector/block/whatever. But i-nodes are too small for that.

> putting that data inside the directory itself,

Now you're talking about directories, which are different from i-nodes. And 
no, if you put the data in the directory, then you couldn't have multiple 
links to a small file.

> Then again, I've sometimes wondered what would happen if you had some 
> filesystem that split the disk into several seperate regions with 
> different block sizes, and allocated files accordingly. (I.e., put the 
> really huge files in the area with big blocks, and the tiny files in 
> some area with tiny block sizes.) I rather suspect you'd permanently be 
> running out of whichever size you happen to need the most tho...

And interesting thought. I've never seen that done.  Given that disks are 
broken into sectors all the same size, and given that the only reasons for 
allocating space in units larger than one sector are defragmentation and 
efficiency of storing pointers to clusters, there's no real good reason for it.

On the other hand, the Amiga formatted the floppy track every time it wrote 
the track, so you could probably actually fit more large files on a disk 
than small files, even if every small file was exactly one sector, by making 
the sectors physically larger on tracks where they store a big file.


-- 
   Darren New, San Diego CA, USA (PST)
   I ordered stamps from Zazzle that read "Place Stamp Here".

Post a reply to this message

Darren New schrieb:
>> Then again, I've sometimes wondered what would happen if you had some 
>> filesystem that split the disk into several seperate regions with 
>> different block sizes, and allocated files accordingly. (I.e., put the 
>> really huge files in the area with big blocks, and the tiny files in 
>> some area with tiny block sizes.) I rather suspect you'd permanently 
>> be running out of whichever size you happen to need the most tho...
> 
> And interesting thought. I've never seen that done.  Given that disks 
> are broken into sectors all the same size, and given that the only 
> reasons for allocating space in units larger than one sector are 
> defragmentation and efficiency of storing pointers to clusters, there's 
> no real good reason for it.

You are aware that modern file systems use block sizes /significantly/ 
larger than the disk sector size?

A disk sector is 512 byte in size virtually everywhere, while file 
systems typically use block sizes one order of magnitude larger.

Why? Because it is actually more memory-efficient to /not/ use even the 
smallest gaps - because that inflates the required management overhead, 
severely reducing the total payload capacity when the files are 
sufficiently /large/ on average.

In the end, some compromise is used, based on the statistical 
distribution of file sizes. A really /good/ system administrator might 
tune the various volumes on his systems to have block sizes that match 
the actual use.

> On the other hand, the Amiga formatted the floppy track every time it 
> wrote the track, so you could probably actually fit more large files on 
> a disk than small files, even if every small file was exactly one 
> sector, by making the sectors physically larger on tracks where they 
> store a big file.

That won't work for hard disk drives: Even if you /could/ still 
low-level-format them (and maybe that's actually still possible with 
special tools), it would be a particularly bad idea, given that they 
don't even disclose their actual drive geometry anymore these days 
(aside from the /official/ total capacity - but even that may be only 
half the truth, as I heard say that modern hard drives reserve some 
sectors as spare, to deal with sectors that over time begin to "almost 
lose data", i.e. become seriously difficult to read - so those can be 
avoided and operation can safely continue without any true problems 
while the system administrator orders a new drive - provided he has kept 
an eye on the SMART status of his drives).

Post a reply to this message

clipka wrote:
> Darren New schrieb:
>>> Then again, I've sometimes wondered what would happen if you had some 
>>> filesystem that split the disk into several seperate regions with 
>>> different block sizes, and allocated files accordingly. (I.e., put 
>>> the really huge files in the area with big blocks, and the tiny files 
>>> in some area with tiny block sizes.) I rather suspect you'd 
>>> permanently be running out of whichever size you happen to need the 
>>> most tho...
>>
>> And interesting thought. I've never seen that done.  Given that disks 
>> are broken into sectors all the same size, and given that the only 
>> reasons for allocating space in units larger than one sector are 
>> defragmentation and efficiency of storing pointers to clusters, 
>> there's no real good reason for it.
> 
> You are aware that modern file systems use block sizes /significantly/ 
> larger than the disk sector size?

Thats what I said.

> A disk sector is 512 byte in size virtually everywhere, while file 
> systems typically use block sizes one order of magnitude larger.
> 
> Why? Because it is actually more memory-efficient to /not/ use even the 
> smallest gaps - because that inflates the required management overhead, 
> severely reducing the total payload capacity when the files are 
> sufficiently /large/ on average.

I take it you didn't actually read what I wrote?

"The only reasons for allocating space larger than one sector are 
defragmentation and efficiency of storing pointers to clusters."

Incidentally, the management overhead isn't that high when the structure you 
use to store the location of files isn't O(n) in the number of clusters. 
NTFS, for example, stores file extents: an unfragmented file will occupy the 
same amount of space to describe its location regardless of file size.

> In the end, some compromise is used, based on the statistical 
> distribution of file sizes. A really /good/ system administrator might 
> tune the various volumes on his systems to have block sizes that match 
> the actual use.

Yep. Or you just buy a bigger drive. :-)

>> On the other hand, the Amiga formatted the floppy track every time it 
>> wrote the track, so you could probably actually fit more large files 
>> on a disk than small files, even if every small file was exactly one 
>> sector, by making the sectors physically larger on tracks where they 
>> store a big file.
> 
> That won't work for hard disk drives:

Hence my specification that it was only Amiga floppy drives, yes. Indeed, 
even other controllers wouldn't do it efficiently, requiring two rotations 
of the disk to first format then write it. The Amiga used the DSP hardware 
(if you want to call it that) to do the MFM encoding, so it could 
efficiently encode the track every time it wrote it without hardware 
dedicated just to that.


> provided he has kept 
> an eye on the SMART status of his drives).

Google says this doesn't work nearly as well as you might like it to. :-)


-- 
   Darren New, San Diego CA, USA (PST)
   I ordered stamps from Zazzle that read "Place Stamp Here".

Post a reply to this message

Darren New schrieb:
> I take it you didn't actually read what I wrote?

Well, I did, but I may have perfectly misunderstood what you had written.

Post a reply to this message

clipka wrote:
> Darren New schrieb:
>> I take it you didn't actually read what I wrote?
> 
> Well, I did, but I may have perfectly misunderstood what you had written.

Fair enough. :-)

-- 
   Darren New, San Diego CA, USA (PST)
   I ordered stamps from Zazzle that read "Place Stamp Here".

Post a reply to this message

Thank you all for your input, friends, though this comes a bit late. Work 
got in the way yesterday, so I could not access the newsgroup. I gained some 
valuable insights into ext2.



Since some of you asked what I did not like about ADS on NTFS, here it is:



There is no easy way to access ADS in Windows Explorer. The only way to do 
it is use API-calls to access the ADS (to my best knowledge).



Again to my best knowledge (which is somewhat lacking in the field of file 
systems, admittedly), you have to scan every single directory using 
API-calls to determine which ADS-files are stored on your HDD.



And most importantly, if you store data in ADS, it is nowhere reported in 
Windows Explorer. I have heard it said that you could run into serious 
problems if you HDD seems to be almost full - but in truth >is< completely 
full, because of ADS-files which file-sizes are reported nowhere.



So, if you are in the mood, you could fill a seemingly empty HDD by creating 
a few really huge ADS-files on  it.



Anybody wanting to write to this HDD should be ready for an interesting 
experience...



I hope linux ext2 does not do the same.

Post a reply to this message