 |
|
|
|
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> You might notice a difference in disk usage if you have enormous
> amounts of small files. (In a non-compacting FS every file takes a
> multiple of 4 kB, rounded up. With ReiserFS they can take less. For
> example if you have 1000 files, each one 128 bytes in length, instead
> of taking 4 MB of disk space they only take 128 kB.)
Interestingly enough, in NTFS, if all the data fits in the same sector
as the inode[1], it goes into the same sector.
> There might be some file systems which support additional permissions,
> but I don't remember right now.
There are, and yes, they aren't especially well supported, especially by
things like rsync, svn, backup programs, etc.
[1] For the NTFS definition of inode.
--
Darren New / San Diego, CA, USA (PST)
Helpful housekeeping hints:
Check your feather pillows for holes
before putting them in the washing machine.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Just some more timings, for S&G:
362000 files, 77G used of 215G, ext3 IDE - 7 minutes 20 seconds.
428000 files, 6.5G used of 82G - ext3 IDE - 2:40
100,000 files, 24G used, NTFS IDE - 2:20
90,000 files, 160G used, NTFS IDE - 1:30.
Really all over the board. It looks like the biggest factor for ext3 is
the capacity of the drive, and I *suspect* the biggest complexity in
NTFS is the complexity of the directory structure. (The 3rd one has a
much more complex directory structure right now than the fourth one).
--
Darren New / San Diego, CA, USA (PST)
Helpful housekeeping hints:
Check your feather pillows for holes
before putting them in the washing machine.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New wrote:
>
> I'd like to find a file system I can reliably use from OSX, Windows, and
> Linux, myself. OSX can almost handle NTFS (as in, it seems to be able to
> read/write NTFS as long as you don't hit the parts that aren't
> supportable with the UNIX file system semantics), but as far as I can
> tell, NTFS still isn't safely writable from Linux? That's what google is
> telling me, at least.
>
Check out NTFS-3G:
http://www.ntfs-3g.org/
--
Eero "Aero" Ahonen
http://www.zbxt.net
aer### [at] removethis zbxtnetinvalid
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
>> You might notice a difference in disk usage if you have enormous
>> amounts of small files. (In a non-compacting FS every file takes a
>> multiple of 4 kB, rounded up. With ReiserFS they can take less. For
>> example if you have 1000 files, each one 128 bytes in length, instead
>> of taking 4 MB of disk space they only take 128 kB.)
>
> Interestingly enough, in NTFS, if all the data fits in the same sector
> as the inode[1], it goes into the same sector.
...which the result that if you have many hundreds of tiny files in a
single directory... the directory structure becomes rather large and
slow to traverse. ;-) (Still better than wasting half your harddrive
though!)
>> There might be some file systems which support additional permissions,
>> but I don't remember right now.
>
> There are, and yes, they aren't especially well supported, especially by
> things like rsync, svn, backup programs, etc.
I thought that would be problematic. ;-) But it at least exists though?
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Orchid XP v8 wrote:
>> Interestingly enough, in NTFS, if all the data fits in the same sector
>> as the inode[1], it goes into the same sector.
>
> ....which the result that if you have many hundreds of tiny files in a
> single directory... the directory structure becomes rather large and
> slow to traverse. ;-) (Still better than wasting half your harddrive
> though!)
Are you joking? Or are you serious? Because from what I understand, the
directory structures have nothing in them but names and pointers into
the master file table (which stores the file records that are the
equivalent of inodes). In other words, the directory hasn't anything to
do with the files, and doesn't get bigger just because you're storing
small files.
I could be wrong about that, tho, if you actually have information that
says otherwise.
> I thought that would be problematic. ;-) But it at least exists though?
On some file systems, yes.
--
Darren New / San Diego, CA, USA (PST)
Helpful housekeeping hints:
Check your feather pillows for holes
before putting them in the washing machine.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Eero Ahonen wrote:
> Check out NTFS-3G:
> http://www.ntfs-3g.org/
Very cool! Thank you!
--
Darren New / San Diego, CA, USA (PST)
Helpful housekeeping hints:
Check your feather pillows for holes
before putting them in the washing machine.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
>>> Interestingly enough, in NTFS, if all the data fits in the same
>>> sector as the inode[1], it goes into the same sector.
>>
>> ....which the result that if you have many hundreds of tiny files in a
>> single directory... the directory structure becomes rather large and
>> slow to traverse. ;-) (Still better than wasting half your harddrive
>> though!)
>
> Are you joking? Or are you serious? Because from what I understand, the
> directory structures have nothing in them but names and pointers into
> the master file table (which stores the file records that are the
> equivalent of inodes). In other words, the directory hasn't anything to
> do with the files, and doesn't get bigger just because you're storing
> small files.
>
> I could be wrong about that, tho, if you actually have information that
> says otherwise.
The information I heard is that "small" files are stored in-line inside
the directory structure to prevent additional seeks (and wasted
half-blocks). The source I got that from wasn't highly technical, so
this might be a simplification.
--
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Orchid XP v8 wrote:
> The information I heard is that "small" files are stored in-line inside
> the directory structure to prevent additional seeks (and wasted
> half-blocks). The source I got that from wasn't highly technical, so
> this might be a simplification.
The "directory structure" includes more than the directories.
There's a "master file table", which holds all the metadata for each
file, such as its names, what directories it's in, how big it is, the
pointer to the ACL, what clusters it's stored on, when it was created,
where it's backed up, what streams it has and where, the encryption
information, and so on. The MFT is basically an array of file objects,
and each file object is an array of file properties. One of those file
properties *can* be the data of the file.
Then there are directories, which store the names and pointers into the
master file table as B-trees.
If you're familiar with old UNIX style file systems, the "master file
table" holds the equivalent of the i-nodes, and the directories hold the
equivalent of the directories.
Having lots of files in a directory leaves it the same size regardless
of how much data is in the files, because the directories just refer to
offsets in the master file table.
(Now you understand the three passes of chkdsk. :-)
The reason the formatting is fast is the master file table is just a
file, as is the free space map and ACL lists (and boot sector, for that
matter), and they're all dynamically allocated as needed just like any
other file, so it's a pretty constant amount of data written during a
format.
Ext2 pre-allocates all the i-nodes, and hence must actually clear them
(i.e., mark them as not in use). And (presumedly) for speed, it spreads
these i-nodes out across the disk, so there are thousands of seeks to
format a 750G disk.
--
Darren New / San Diego, CA, USA (PST)
Helpful housekeeping hints:
Check your feather pillows for holes
before putting them in the washing machine.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
|
|
