POV-Ray : Newsgroups : povray.off-topic : Why is defragging so slow? Server Time
6 Sep 2024 07:14:54 EDT (-0400)
  Why is defragging so slow? (Message 51 to 60 of 61)  
<<< Previous 10 Messages Goto Latest 10 Messages Next 1 Messages >>>
From: Darren New
Subject: Re: Why is defragging so slow?
Date: 5 Jun 2009 13:49:43
Message: <4a295ab7$1@news.povray.org>
Darren New wrote:
> Yeah! That's exactly what I need on Vista. :-) 

Or not even that, really. If exec() would just read the whole file into RAM 
in one blow, it would probably be 10x as fast. I should be able to start up 
half a gig of executables in well under 10 seconds, and I'm pretty sure I'm 
not loading that much stuff up when I log in. :-)

-- 
   Darren New, San Diego CA, USA (PST)
   There's no CD like OCD, there's no CD I knoooow!


Post a reply to this message

From: Patrick Elliott
Subject: Re: Why is defragging so slow?
Date: 5 Jun 2009 22:54:41
Message: <4a29da71@news.povray.org>
Darren New wrote:
> Patrick Elliott wrote:
>> Simplest answer is, it behaved that way in 3.11, when it was "even 
>> more critical", 
> 
> Huh?  Did MS make a defragmenter for Win3.11? I thought it was just the 
> dos defrag? All the third-party defrags I remember using did a fine job 
> of defragmenting absolutely everything.
> 
There is a relevant difference between DOS Defrag and a Windows one, 
back when Win3.11 "was" basically just DOS with a GUI? lol

>> and MS never bothered to steal the idea of fixing it from the few 
>> companies whose products **did** do it. ;)
> 
> Cite?  Certainly the XP defragmenter moves files around to open up free 
> spaces.  I suspect it's just files that have absolute addresses in them, 
> like the UNC and system restore points.
> 
Hmm. Nope. I had the *identical* issue with 95 and 98. You may be partly 
correct, to some extent, in that "often" the virtual memory file gets 
fragmented to hell too, and the defrag can't move those (unlike Norton 
used to in 3.11->98). So, if "that" file gets hacked up into small bits, 
the defragger will "never" fix them. Running it "over and over and over" 
under 95/98 could get it to "partly" fix some of the gaps, but at some 
point it would just flat out refuse to move anything else around, so you 
had empty spots between them, no matter how many times it ran. MS Defrag 
has "never" changed or improved this behavior in "any" release.

-- 
void main () {
   If Schrödingers_cat is alive or version > 98 {
     if version = "Vista" {
       call slow_by_half();
       call DRM_everything();
     }
     call functional_code();
   }
   else
     call crash_windows();
}

<A HREF='http://www.daz3d.com/index.php?refid=16130551'>Get 3D Models, 
3D Content, and 3D Software at DAZ3D!</A>


Post a reply to this message

From: Darren New
Subject: Re: Why is defragging so slow?
Date: 5 Jun 2009 22:58:37
Message: <4a29db5d$1@news.povray.org>
Patrick Elliott wrote:
> There is a relevant difference between DOS Defrag and a Windows one, 
> back when Win3.11 "was" basically just DOS with a GUI? lol

Yes, lol. Because you ran the defrag under DOS, nothing else ran at the same 
time, lol.

>>> and MS never bothered to steal the idea of fixing it from the few 
>>> companies whose products **did** do it. ;)
>>
>> Cite?  Certainly the XP defragmenter moves files around to open up 
>> free spaces. 
>>
> Hmm. Nope. 

OK. I guess you don't know what "cite" means. :-)

-- 
   Darren New, San Diego CA, USA (PST)
   There's no CD like OCD, there's no CD I knoooow!


Post a reply to this message

From: Patrick Elliott
Subject: Re: Why is defragging so slow?
Date: 5 Jun 2009 23:07:29
Message: <4a29dd71$1@news.povray.org>
Warp wrote:
> Darren New <dne### [at] sanrrcom> wrote:
>> Everyone uses the same API. If there's a reason the files can't be moved, no 
>> program is going to be able to move them.
> 
>   MyDefrag was able to move those strange clusters, even though it uses the
> regular defragmenter system API to do so...
> 
Ah, but see... They used it in a "non-standard" way. In all likelihood, 
with Win7 they will release a new .DEFRAG API, which ***accidentally*** 
breaks this non-standard method, while offering some crazy ass 
alternative, like using another drive to "automatically" tranfer files, 
temporarily, so they are defragged when copied back (will only work if 
you have multiple drives/partitions), kind of like how they provides 
some new ways in .NET to "fake" access to the old IUNKNOWN functions in 
the ActiveX API, which where never properly documented, and couldn't be 
used easily, thereby letting you hot-load new .NET objects, and allow 
your application to use them without "hard coding" the interfaces into 
your application before hand, but which also, even more completely, 
prevents you using them in the "design mode" that is only available to 
you when "building" an interface in Visual Studio.

See, there are plenty of documents that, and I love this, tell you how 
to "make" controls that "respond to" design mode, so you can better 
handle them in Visual Studio, but... gods forbid someone wants to make 
an application of their own that took advantage of that feature, so you 
could "design" an interface, and say.. save it to a layout document, to 
be reloaded later, when actually using the new interface. Your only 
"allowed" to do that via the Visual Studio. And now, you can't even 
"try" to do it without that, because the new .NET buries the interfaces 
and APIs needed to even attempt it under 50 new layers of BS (while 
giving you half of what you wanted in the first place).

So, basically, figure that if this is a feature people will want, its 
something MS will provide "half" of the function for in the new API, 
while simultaneously making it completely impossible to use the new API 
to get "all" of the functionality any more. ;) lol

-- 
void main () {

     if version = "Vista" {
       call slow_by_half();
       call DRM_everything();
     }
     call functional_code();
   }
   else
     call crash_windows();
}

<A HREF='http://www.daz3d.com/index.php?refid=16130551'>Get 3D Models, 
3D Content, and 3D Software at DAZ3D!</A>


Post a reply to this message

From: Patrick Elliott
Subject: Re: Why is defragging so slow?
Date: 5 Jun 2009 23:16:39
Message: <4a29df97$1@news.povray.org>
Darren New wrote:
> Patrick Elliott wrote:
>> There is a relevant difference between DOS Defrag and a Windows one, 
>> back when Win3.11 "was" basically just DOS with a GUI? lol
> 
> Yes, lol. Because you ran the defrag under DOS, nothing else ran at the 
> same time, lol.
> 
>>>> and MS never bothered to steal the idea of fixing it from the few 
>>>> companies whose products **did** do it. ;)
>>>
>>> Cite?  Certainly the XP defragmenter moves files around to open up 
>>> free spaces.
>> Hmm. Nope. 
> 
> OK. I guess you don't know what "cite" means. :-)
> 
You want me to what? Find a web page written by someone even more 
annoyed by the behavior than I am to give a jaded statement to the same 
effect? lol Seriously though, I don't dispute that it "does" move things 
around. What it doesn't do is "move them if it doesn't see any reason 
to, and consolidating free space isn't always, apparently, a 'reason'". 
And it flat out refuses to move things that it "could", if the bothered 
to put in some way to copy a block of an "in use" file, and.. well do 
the stuff that defraggers do to every other file anyway. I mean, can you 
seriously tell me that they couldn't come up with "some" way to be able 
to, I don't know, write a block of memory to some other part of the 
virtual file, then invalidate the part being moved, then revert again 
when done, or "something"?

And, the idea that DOS based defraggers where the only ones in 3.11 
isn't "entirely" accurate. The Norton one came up during 3.11, and it 
"did" defrag "everything", at that time, including the virtual memory 
file, and just about anything and everything else "except" for a small 
handful of active files in the OS. Say, 90% of all the stuff that no 
defragger today will even touch, for the most part.

-- 
void main () {
   If Schrödingers_cat is alive or version > 98 {
     if version = "Vista" {
       call slow_by_half();
       call DRM_everything();
     }
     call functional_code();
   }
   else
     call crash_windows();
}

<A HREF='http://www.daz3d.com/index.php?refid=16130551'>Get 3D Models, 
3D Content, and 3D Software at DAZ3D!</A>


Post a reply to this message

From: Darren New
Subject: Re: Why is defragging so slow?
Date: 6 Jun 2009 00:19:38
Message: <4a29ee5a@news.povray.org>
Patrick Elliott wrote:
> You want me to what? Find a web page written by someone even more 
> annoyed by the behavior than I am to give a jaded statement to the same 
> effect?

I rest my case.

> Seriously though, I don't dispute that it "does" move things 
> around. What it doesn't do is "move them if it doesn't see any reason 
> to, and consolidating free space isn't always, apparently, a 'reason'".

The XP defragger usually consolodated much free space for me. Not 100%, but 
certainly some.

> And it flat out refuses to move things that it "could", if the bothered 
> to put in some way to copy a block of an "in use" file, 

I'm glad you know better than Microsoft how easy it is to do that sort of thing.

> and.. well do 
> the stuff that defraggers do to every other file anyway. I mean, can you 
> seriously tell me that they couldn't come up with "some" way to be able 
> to, I don't know, write a block of memory to some other part of the 
> virtual file, then invalidate the part being moved, then revert again 
> when done, or "something"?

I don't even know what you're talking about there.

> And, the idea that DOS based defraggers where the only ones in 3.11 
> isn't "entirely" accurate. The Norton one came up during 3.11, and it 
> "did" defrag "everything", at that time, including the virtual memory 
> file, and just about anything and everything else "except" for a small 
> handful of active files in the OS. Say, 90% of all the stuff that no 
> defragger today will even touch, for the most part.

FAT is a lot easier to defrag than NTFS, yes. But feel free to continue 
ranting while cackling gleefully. ;-)

-- 
   Darren New, San Diego CA, USA (PST)
   There's no CD like OCD, there's no CD I knoooow!


Post a reply to this message

From: clipka
Subject: Re: Why is defragging so slow?
Date: 6 Jun 2009 07:25:00
Message: <web.4a2a5122b1ff837af708085d0@news.povray.org>
Warp <war### [at] tagpovrayorg> wrote:
>   The other, lighter defragger is behaving the exact same way: It's just
> refusing to move these small files, and consequently failing to defrag the
> one huge file. No reason is given in either the visual representation of the
> drive (all these tiny files are colored as regular, movable data) nor the
> analysis reports for this.

>   I can't understand why both programs are doing this.

Sounds like there's some serious reason to not touch them. Maybe they're
currently open whenever you run defrag?

Tried booting in safe mode and then defragging? That should reduce the number of
processes running, and thus the number of open files.


Post a reply to this message

From: clipka
Subject: Re: Why is defragging so slow?
Date: 6 Jun 2009 08:20:00
Message: <web.4a2a5e08b1ff837af708085d0@news.povray.org>
Warp <war### [at] tagpovrayorg> wrote:
>   Why does it take such a humongous amount of time? I can't understand.

As someone already said, head movements is the limiting factor here.

Once the defrag software has made up its mind where to ultimately place the
files, the remaining time it will spend...

- picking up clusters from their original position (remember, the files start
heavily fragmented, so *many* major head movements to be expected here already)

- copying those clusters to their target position (or, more frequently, an
interim position) far away (another major head movement; fortunately, we're
talking about contiguous clusters now)

- updating the File Allocation Tables (there's typically two of them) at the
start of the disk (one more major head mvoement), to reflect the changes

- doing roughly the same to move clusters from interim positions to their target
position (fortunately, they should be contiguous already by now).


The speed of a defrag software will heavily depend on how good it is at
minimizing head movement; for instance, if for some reason clusters of a file
happen to be ordered wrong way round (e.g. C...A...B...D), a naive defrag
software may try to pick them up in logical order, while an intelligent one
could recognize the situation and pick them up in physical order instead.

Similarly, a naive software may move clusters out of the way to the very end of
the disk - while an intelligent one may temporarily store some clusters even
where other clusters are intended to end up, and just make sure the spot is
free again soon enough.


Having lots of physical main memory installed should help, because a bigger
number of clusters can be processed at once, and at least head seeks for
writing the clusters back and updating the FAT will be reduced.

Having a disk with >50% free space helps a lot, too - otherwise clusters may
have to be moved back and forth multiple times before reaching their final
destination.

For some really hard cases, it may be worth considering to mirror the whole
disk, delete all the (movable) files from the original one (remember to empty
the recycle bin), then copy those files back from the mirror. If we're talking
about an OS partition, doing this on a different machine may work (in case it
doesn't, you can still do an exact restore from the mirror).

Placing applications, games and user data on a partition separate from the OS is
a very helpful thing, too.


>   I also noticed that both defragmenters failed to use the free space
> in the partition (over 80 GB) to their advantage, and instead moving
> small clusters around one at a time. I'm pretty certain that if all the
> free available space was used to copy data, the whole process could be
> a lot faster.

Maybe the defrag software was actually able to move most clusters directly to
their intended destination, without having to temporarily store them somewhere
else. Or it chose temporary storage location the smart way, as close to the
source or intended destination as possible, to minimize head movements.


Post a reply to this message

From: Chambers
Subject: Re: Why is defragging so slow?
Date: 6 Jun 2009 14:16:19
Message: <4a2ab273$1@news.povray.org>
Tim Cook wrote:
> Norton's defragger, at least, used to have a 'defragment free space' 
> checkbox.  No idea why that wasn't a default setting, or why almost no

Because if you compact all files, and then grow a file at all, you have 
two options: use the free space at the end, or fragment the file.

If you don't want to rewrite the whole file, fragmenting it is (in that 
situation) the easier solution.

-- 
Chambers


Post a reply to this message

From: Patrick Elliott
Subject: Re: Why is defragging so slow?
Date: 6 Jun 2009 14:39:51
Message: <4a2ab7f7$1@news.povray.org>
Darren New wrote:
>> And it flat out refuses to move things that it "could", if the 
>> bothered to put in some way to copy a block of an "in use" file, 
> 
> I'm glad you know better than Microsoft how easy it is to do that sort 
> of thing.
> 
Not actually knowing how their mess works, I have no real idea what road 
blocks they may have created to make it impossible, but at the same 
time, we are talking about Microsoft here... The people that only 
"recently" fixed certain bugs in Excel and other things, which have been 
around since pretty much when the product "first" came out too. Tell me, 
is the "some years Windows doesn't recognize one day out of the year, 
because its code for most DBs/Spreadsheets/anything using dates of 
calenders, is hard coded to think there will never be 4 Saturdays in a 
month, fixed yet? And how long has that been around? lol

See, when some random 3rd party application, like MyDefrag can manage, 
using the same API, what Microsoft has failed to, one has to wonder if 
MS even *cares* that their version is, and has been, basically broken 
from day one.

-- 
void main () {
   If Schrödingers_cat is alive or version > 98 {
     if version = "Vista" {
       call slow_by_half();
       call DRM_everything();
     }
     call functional_code();
   }
   else
     call crash_windows();
}

<A HREF='http://www.daz3d.com/index.php?refid=16130551'>Get 3D Models, 
3D Content, and 3D Software at DAZ3D!</A>


Post a reply to this message

<<< Previous 10 Messages Goto Latest 10 Messages Next 1 Messages >>>

Copyright 2003-2023 Persistence of Vision Raytracer Pty. Ltd.