 |
|
|
|
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New schrieb:
> clipka wrote:
>> Darren New schrieb:
>>
>>>> (Not developed by MS themselves, btw.)
>>>
>>> That may have been true a long time ago, but I don't expect it's true
>>> any more.
>>
>> Vice versa actually: They used to develop their own defrag,
>
> No, they used Executive Software's DiskKeeper stuff. MS developed all
> the APIs and such. Executive Software just wrote the user interface part
> (and of course provided info on how the API should be designed). I'm
> pretty sure nobody outside MS wrote software that frobs the NTFS file
> system to rearrange blocks.
"Diskeeper is a file system defragmenter originally for the VAX series
of minicomputers and later released for Microsoft Windows. It is the
flagship product of Diskeeper Corporation (formerly Executive Software).
The defragmenter program included with the Windows 2000, 2003, and XP
operating systems is based on a basic version of a previous Diskeeper
version."
(Wikipedia)
While the low-level code to perform the actual /swapping/ of individual
blocks of NTFS may well be MS stuff, /everything/ on top - including
such essential decisions as where to put which block of a file - is up
to individual software tools, and if MS did write anything of that which
is included in Windows XP, then apparently it's only the UI.
According to Wikipedia, the defragmenter that shipped with Windows 9x
was no MS stuff either, but licensed from Symantec.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
clipka wrote:
> While the low-level code to perform the actual /swapping/ of individual
> blocks of NTFS may well be MS stuff, /everything/ on top - including
> such essential decisions as where to put which block of a file - is up
> to individual software tools,
Yes. But that's really pretty trivial stuff compared to actually writing
the code in the file system that makes it work.
Executive Software wrote the code to do the swapping on the VAX, too. I.e.,
they wrote *all* the code, not just the code to invoke the API to defragment
a file.
> and if MS did write anything of that which
> is included in Windows XP, then apparently it's only the UI.
If MS wrote the UI for XP, then everything is written by MS, since the UI
for XP just makes the decisions about where to put the blocks of the files
and swaps them. I.e., if MS wrote the UI for XP, and the algorithm for
picking which files to move where, and the API for swapping the blocks, then
they wrote everything, right?
I'm not sure I'm following. I'm just talking about it having two parts - the
UI, and the file system operations.
> According to Wikipedia, the defragmenter that shipped with Windows 9x
> was no MS stuff either, but licensed from Symantec.
Yes.
--
Darren New, San Diego CA, USA (PST)
I ordered stamps from Zazzle that read "Place Stamp Here".
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New schrieb:
> I'm not sure I'm following. I'm just talking about it having two parts -
> the UI, and the file system operations.
No, it's actually three parts:
(1) The UI to pick which drive to defragment, to visualize
defragmentation status, and to show a report afterwards
(2) The decision-making where to actually put exactly which file (the
"business logic" so to speak)
(3) The low-level code to swap one particular block with another in a
safe way
I'm not too familiar with NTFS, but I expect (3) to be not too
complicated, even if it may not be as simple as with FAT partitions,
while (2) is the one that actually determines the quality of the
defragmentation result: For instance, whether files in the same
directory are kept close together, or whether they are grouped according
to other criteria; how much free space is inserted after each file; and
plenty more stuff like that.
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
clipka wrote:
> (1) The UI to pick which drive to defragment, to visualize
> defragmentation status, and to show a report afterwards
OK. I found this too trivial to talk about, given how simple the UI was.
> (2) The decision-making where to actually put exactly which file (the
> "business logic" so to speak)
Yeah. Really, it isn't that hard. If you're to the point where you have
movable files that you don't just move, your disk is too full anyway. The
only hard part is guessing the future of the files and how they'll be used
after you defrag them, and tracking all *that* is the responsibility of
other components of the OS.
> (3) The low-level code to swap one particular block with another in a
> safe way
> I'm not too familiar with NTFS, but I expect (3) to be not too
> complicated,
Until you take into account that other processes are reading and writing the
blocks at the same time, and you have to recover from a crash at any time
without actually running a chkdsk, and that some blocks are buffered in
memory, and some blocks are buffered on the readyboost memory, and that you
have volume shadow copies, and that you have compressed files, and that
defragging a file can actually change its size... But sure, if you ignore
all the hard parts, it's pretty easy.
Go read the papers Executive Software wrote about it. It is *far* from
trivial, and its even more complicated in Vista.
> while (2) is the one that actually determines the quality of the
> defragmentation result: For instance, whether files in the same
> directory are kept close together, or whether they are grouped according
> to other criteria; how much free space is inserted after each file; and
> plenty more stuff like that.
Sure. Plus, of course, MS added all the stuff that keeps track of how often
you use the file, what order the blocks are used in, and so on. None of that
is part of the defrag system at all.
For example, Vista tracks what blocks of which files are demand-paged in as
you boot and log in. (That's the "readyboot" subsystem.) Their defragger
then doesn't defrag the file in block order, but rather into the order you
tend to demand-page it in, if I understand correctly.
You can make it arbitrarily complicated. Most UIs don't. DiskKeeper only
defragged stuff that it guessed would be faster to defrag than to just read
fragmented.
--
Darren New, San Diego CA, USA (PST)
I ordered stamps from Zazzle that read "Place Stamp Here".
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New wrote:
> Patrick Elliott wrote:
>> it will flat out *refuse* to consolidate the free space, even when
>> there is no sane reason to leave a handful of files scattered willy
>> nilly over the remaining disk space.
>
> There is a sane reason, actually. You just don't know what that reason
> is. :-) One possibility is that they're protected because they're
> branches of files carrying around the encryption key information.
> Another possibility is that they're locked open for writing.
>
No there isn't. Often, re-running the defrag will result in them moving
them anyway. The problem is simply that, unlike several third party
defraggers, the one in Windows **doesn't** consolidate free space, only
files. Once the files themselves are no longer fragmented, or there is
no available space, with the current configuration on the drive, to move
one of them, it stops. This happens **even if** it could have
consolidated some of the free space and thus "gained" the space needed
to move the file it refused to defragment. I know, I have played with
the thing enough, and re-run in multiple times, so I am well aware what
it will and won't do. And it simply won't move some files (even if it
clearly labels them, in its display, as "movable", if it doesn't need to
do so, since it doesn't treat "free space" as fragmented, and in need of
correction.
The stupid thing about this is.. assuming the files are mostly stable,
and won't change, like the OS partition, it actually makes it "more"
likely that any new files installed will "become" fragmented, the moment
you write them, since all the free space gaps are still there.
>> And, that is the problem. If you need to defrag something like the
>> page file, during boot, how do you do that, if it won't move the
>> stupid files that are in the way? :(
>
> Copy those files to a different disk, delete them, defrag, copy them back.
>
Ah yes.. Because it "tells you" which files they are, so you can do
that, and they will never be something critical, which you can't move
someplace else first. And, as I pointed out above, **this doesn't fix
anything*, since it won't consolidate free space.
>>> The only XP memory limits I know of was running into the 4G boundary.
>>> And apparently that's a licensing thing - seems XP x86 is happy to
>>> use however much memory you put in there, except that Microsoft tells
>>> it not to.
>>>
>> Yeah. I know. But, the hack requires changing the boot.sys file.
>
> Different hack, I suspect.
>
Don't think so... Though, maybe there is some system file you can
replace, or something, that does it automatically. All I know is that
one version involves adding a flag to the startup for the OS, and that
is, I think, in the boot.sys file.
>> They managed to make the OS smart enough to boot anyway, if its
>> damaged/missing (mine is missing for some reason), but didn't include
>> any way to rebuild it, if you lost/mangled it. So.. No file, no clue
>> how to write it, and therefor, no means to override the setting.
>
> LMGTFY. http://mirror.href.com/thestarman/asm/mbr/bootini.htm
>
Uh, yeah. And I am assuming my system is vanilla enough I could do that
manually, but I would still prefer something that could look at my
existing install and just recreate the file, so I know it is correct. I
have no clue what any of the partition, etc. stuff is in there, and I
don't want to spend an hour figuring it out. ;)
--
void main () {
If Schrödingers_cat is alive or version > 98 {
if version = "Vista" {
call slow_by_half();
call DRM_everything();
}
call functional_code();
}
else
call crash_windows();
}
<A HREF='http://www.daz3d.com/index.php?refid=16130551'>Get 3D Models,
3D Content, and 3D Software at DAZ3D!</A>
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New wrote:
> Patrick Elliott wrote:
>> that are in the way? :(
>
> I can show you how to get rid of most or all of those, btw.
>
> It's probably either:
> 1) Encrypted file system data forks,
> 2) volume shadow copies (i.e., system restore files),
> 3) the USN journal
> none of which can be moved by the normal defrag mechanism.
>
As I said, The files its not moving are marked as "movable" Its just not
defragging the free space, beyond a very minimalistic amount needed for
consolidating the files it is defragging, so you get gaps anyway.
--
void main () {
If Schrödingers_cat is alive or version > 98 {
if version = "Vista" {
call slow_by_half();
call DRM_everything();
}
call functional_code();
}
else
call crash_windows();
}
<A HREF='http://www.daz3d.com/index.php?refid=16130551'>Get 3D Models,
3D Content, and 3D Software at DAZ3D!</A>
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Darren New wrote:
> clipka wrote:
>> soon as you expect the contents of the files to change, it will
>> actuall be more efficient to have some free space after each file, to
>> allow for it to grow without fragmenting /again/.
>
> Interestingly enough, Windows actually does keep track of which files
> change and which don't. Plus, it's probably pretty easy to guess in most
> cases. EXE and DLL files? Pack em tight. Mailbox files? You probably
> want to leave some space.
>
Ok, point taken, but.. 25% of the disk, so that the one it refused to
move is smack in the middle of the part I need to use? lol Rather have
it move all the ones that are likely to change to the end of the
contiguous area, so that the likely amount of fragmentations is less,
while still leaving the *end* of the disk free of random files. But,
maybe I am just overly picky... :p
--
void main () {
If Schrödingers_cat is alive or version > 98 {
if version = "Vista" {
call slow_by_half();
call DRM_everything();
}
call functional_code();
}
else
call crash_windows();
}
<A HREF='http://www.daz3d.com/index.php?refid=16130551'>Get 3D Models,
3D Content, and 3D Software at DAZ3D!</A>
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
SharkD wrote:
> Patrick Elliott wrote:
>> On a semi-side note. Had to defrag mine, which Windows doesn't
>> normally allow. Page Defrag utility from sysinternals works nice
>> (though I need to run it again, now that I succeeded in defragging all
>> the files I couldn't, because the page file was in 33,000 pieces...
>> O.o It runs before anything but the basics starts up, just after the
>> initial loading screen, so pagefile.sys isn't yet in use.
>>
>> You would think, even with the slight overhead, trying to keep this
>> thing in one chunk, as much as possible, would have been useful... Sigh!
>
> You can use the defrag utility if you restart using Safe Mode + Command
> Prompt. This might allow you to defrag the page file since lots of stuff
> is being bypassed.
>
> -Mike
Hmm. If it was that simple, I doubt they would have developed the tool,
but maybe. Though, in the case of this tool, its designed to *only*
defrag the one file, so its faster, and as a boot process, you might
probably want that. ;)
--
void main () {
If Schrödingers_cat is alive or version > 98 {
if version = "Vista" {
call slow_by_half();
call DRM_everything();
}
call functional_code();
}
else
call crash_windows();
}
<A HREF='http://www.daz3d.com/index.php?refid=16130551'>Get 3D Models,
3D Content, and 3D Software at DAZ3D!</A>
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Patrick Elliott wrote:
> No there isn't. Often, re-running the defrag will result in them moving
> them anyway.
I don't know if you're talking XP or Vista, but there are things in Vista
that are locked open until you modify enough stuff on disk that it rolls out
of the cache and gets closed. System snapshots spring to mind.
> The problem is simply that, unlike several third party
> defraggers, the one in Windows **doesn't** consolidate free space, only
> files.
Bull. You can watch it happen yourself. It shows up in the status line of
the GUI on XP, for example.
> This happens **even if** it could have
> consolidated some of the free space and thus "gained" the space needed
> to move the file it refused to defragment.
This I'll believe. But that just means it doesn't move things reactively.
That doesn't mean it doesn't move things proactively.
> I know, I have played with
> the thing enough, and re-run in multiple times, so I am well aware what
> it will and won't do. And it simply won't move some files (even if it
> clearly labels them, in its display, as "movable", if it doesn't need to
> do so, since it doesn't treat "free space" as fragmented, and in need of
> correction.
No, it doesn't defrag all free space. But it does defrag some free space.
There are calculations done as to whether it's worthwhile to move the file
that's already defragmented.
> The stupid thing about this is.. assuming the files are mostly stable,
> and won't change,
Yep. Bad assumption, of course. But as I said earlier, the problem is with
guessing the future.
> Ah yes.. Because it "tells you" which files they are, so you can do
> that, and they will never be something critical, which you can't move
> someplace else first. And, as I pointed out above, **this doesn't fix
> anything*, since it won't consolidate free space.
So write something that tells you what they are, and that moves it for you.
You have the APIs. :-)
Seriously, there should be a piece of code that says "find the files with a
gap more than 3x the size of the file in front and move it. :)
>>>> The only XP memory limits I know of was running into the 4G
>>>> boundary. And apparently that's a licensing thing - seems XP x86 is
>>>> happy to use however much memory you put in there, except that
>>>> Microsoft tells it not to.
>>>>
>>> Yeah. I know. But, the hack requires changing the boot.sys file.
>>
>> Different hack, I suspect.
>>
> Don't think so...
Well, considering the OP had to hack the registry and turn off a couple of
start-up tasks, I'm pretty sure. You're talking about the 2G vs 3G limit.
I'm talking about making 32-bit XP use 8G of RAM.
> Uh, yeah. And I am assuming my system is vanilla enough I could do that
> manually, but I would still prefer something that could look at my
> existing install and just recreate the file, so I know it is correct. I
> have no clue what any of the partition, etc. stuff is in there, and I
> don't want to spend an hour figuring it out. ;)
What partition of which drive is NTLDR on? Plug in those numbers, and you
win, if you're just using plain old SATA or IDE drives with no RAID or any
weird stuff like that. Heck, get the numbers out of your GRUB
configuration for that matter.
--
Darren New, San Diego CA, USA (PST)
I ordered stamps from Zazzle that read "Place Stamp Here".
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Patrick Elliott wrote:
> it move all the ones that are likely to change to the end of the
> contiguous area, so that the likely amount of fragmentations is less,
That's probably a really good idea. Shove all the stuff you know won't
change to the base of the drive, and the other stuff to the middle. Feel
free to implement that algorithm. :-)
Might be fun to come up with a CLI defragger that could tell you where a
file is, where the free space is, or move the file to specific free space.
Then people could play with it with whatever language they want.
--
Darren New, San Diego CA, USA (PST)
I ordered stamps from Zazzle that read "Place Stamp Here".
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
|
|
