On Mon, 12 Nov 2007 19:20:08 -0500, Alain wrote:

> Ever heard of "sparse" files?

Yes.  Old hand at NetWare; didn't know that Windows filesystems supported 
sparse files.

Jim

Post a reply to this message

Jim Henderson wrote:
> On Sun, 11 Nov 2007 15:26:48 -0800, Darren New wrote:
> 
>> More precisely, if you have a fragmented file taking 3 blocks and
>> filling them each only 10%, defragging that file will free two blocks.
> 
> I don't think that can happen with "standard" files in NTFS or FAT 

I don't know about FAT, but I was surprised when I saw a log file that 
actually consumed more clusters than there were lines in the file. WTF?

> would the OS write to a partial block but not the entire block?  Doesn't 
> make sense to me...

Open the file, write a little bit, close it. Open it again, append a 
little, close it. Repeat.  Possibly it requires other people to also 
have the file open at the same time, as for reading or something?

NTFS isn't very good at avoiding fragmentation, methinks.

-- 
   Darren New / San Diego, CA, USA (PST)
     Remember the good old days, when we
     used to complain about cryptography
     being export-restricted?

Post a reply to this message

Jim Henderson wrote:
> On Mon, 12 Nov 2007 19:20:08 -0500, Alain wrote:
> 
>> Ever heard of "sparse" files?
> 
> Yes.  Old hand at NetWare; didn't know that Windows filesystems supported 
> sparse files.

Yah, it does. Well, NTFS does, at least. Indeed, fundamental parts of 
the system (like the USN journal) rely on the fact that you can delete 
blocks off the start of a file and recover the space used therein.

Unlike UNIX, it's not just a matter of seeking around. You have to open 
the file as a sparse file, then you have to say "hey, this area is 
sparse" rather than (say) just writing all zeros to the blocks or something.

-- 
   Darren New / San Diego, CA, USA (PST)
     Remember the good old days, when we
     used to complain about cryptography
     being export-restricted?

Post a reply to this message

Orchid XP v7 wrote:
> The allocation engine allocates a full block to a file, and does not 
> allocate any additional ones until the present ones are full.

How do you know?

-- 
   Darren New / San Diego, CA, USA (PST)
     Remember the good old days, when we
     used to complain about cryptography
     being export-restricted?

Post a reply to this message

On Mon, 12 Nov 2007 20:39:45 -0800, Darren New wrote:

> Jim Henderson wrote:
>> On Mon, 12 Nov 2007 19:20:08 -0500, Alain wrote:
>> 
>>> Ever heard of "sparse" files?
>> 
>> Yes.  Old hand at NetWare; didn't know that Windows filesystems
>> supported sparse files.
> 
> Yah, it does. Well, NTFS does, at least. Indeed, fundamental parts of
> the system (like the USN journal) rely on the fact that you can delete
> blocks off the start of a file and recover the space used therein.
> 
> Unlike UNIX, it's not just a matter of seeking around. You have to open
> the file as a sparse file, then you have to say "hey, this area is
> sparse" rather than (say) just writing all zeros to the blocks or
> something.

Ah, I see - learned something new today. :-)

Jim

Post a reply to this message

On Mon, 12 Nov 2007 20:38:07 -0800, Darren New wrote:

> Open the file, write a little bit, close it. Open it again, append a
> little, close it. Repeat.  Possibly it requires other people to also
> have the file open at the same time, as for reading or something?

Don't think so.  Sparse files I could see, but a proper sparse file 
wouldn't shrink during a defrag, either, based on my own experiences, 
because the "empty" space is as important as the real data is to the 
application.

Jim

Post a reply to this message

Jim Henderson wrote:
> On Mon, 12 Nov 2007 20:38:07 -0800, Darren New wrote:
> 
>> Open the file, write a little bit, close it. Open it again, append a
>> little, close it. Repeat.  Possibly it requires other people to also
>> have the file open at the same time, as for reading or something?
> 
> Don't think so.

I don't know. All I can say is that I've seen files with 2K length 
spread over half a dozen blocks as reported by defrag. :-)  I'm not sure 
what causes it.

Try running the defrag GUI version, look at the report, and see if 
things like your registry hives or event logs are fragmented and how 
much. In my expereince, it's not uncommon.

>  Sparse files I could see, but a proper sparse file 
> wouldn't shrink during a defrag, either, based on my own experiences, 
> because the "empty" space is as important as the real data is to the 
> application.

Yah. Sparse files are already deallocated, so you can't really 
deallocate more.

-- 
   Darren New / San Diego, CA, USA (PST)
     Remember the good old days, when we
     used to complain about cryptography
     being export-restricted?

Post a reply to this message

Darren New wrote:
> I don't know. All I can say is that I've seen files with 2K length 
> spread over half a dozen blocks as reported by defrag. :-)  I'm not sure 
> what causes it.

I mean, 43 fragments for a 1K ntuser.dat?  15 fragments for a 1K 
default.log? In a 4K cluster size?  Clearly, there has to be fragmented 
files with not-fully-packed blocks, if defrag is to be believed.

Maybe that's the problem tho - the layout is right, the reporting is funky.

-- 
   Darren New / San Diego, CA, USA (PST)
     Remember the good old days, when we
     used to complain about cryptography
     being export-restricted?

Post a reply to this message

Attachments:
Download 'image1.gif' (14 KB)

Preview of image 'image1.gif'

> Unlike UNIX, it's not just a matter of seeking around. You have to open 
> the file as a sparse file, then you have to say "hey, this area is 
> sparse" rather than (say) just writing all zeros to the blocks or 
> something.

Not really... Cross-platform programs that don't use any 
Windows-specific API manage to create sparse files here. That's why 
download managers that download in chunks give the option to 
pre-allocate: sparse files that are filled randomly get fragmented quite 
badly.

Post a reply to this message

Darren New escribió:
> I mean, 43 fragments for a 1K ntuser.dat?

That's probably mis-reporting the size. NTUSER.DAT has the whole user 
registry hive. It's 6MB on my machine, no way it can be 1K on yours.

Post a reply to this message