 |
|
|
|
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On Mon, 01 Jun 2009 22:07:26 +0200, Warp <war### [at] tag povrayorg> wrote:
> I have a secondary 180GB FAT32 partition
That would be the biggest problem right there. FAT32 is really horrible on
partitions that large.
> Last time I defragmented this partition was something like 3 years ago
That would be problem number two.
If you absolutely must use FAT32 (compatibility with other operating
systems being the only plausible reason), the fastest way to defrag is to
simply copy everything to another partition (or a spare drive), reformat,
and then copy everything back.
--
FE
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
On Mon, 01 Jun 2009 22:46:49 +0200, Fredrik Eriksson wrote:
> On Mon, 01 Jun 2009 22:07:26 +0200, Warp <war### [at] tagpovrayorg> wrote:
>> I have a secondary 180GB FAT32 partition
>
> That would be the biggest problem right there. FAT32 is really horrible
> on partitions that large.
>
>
>> Last time I defragmented this partition was something like 3 years ago
>
> That would be problem number two.
Concur with both of those ideas. Thing to keep in mind as well is that
(at least IIRC) defragmenters tend to bypass caching mechanisms, so you
lose some performance there as well (compared to doing a file copy, for
example).
Jim
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Fredrik Eriksson <fe79}--at--{yahoo}--dot--{com> wrote:
> On Mon, 01 Jun 2009 22:07:26 +0200, Warp <war### [at] tagpovrayorg> wrote:
> > I have a secondary 180GB FAT32 partition
> That would be the biggest problem right there. FAT32 is really horrible on
> partitions that large.
Why would FAT32 become slow with partitions that large?
And anyways, no matter how slow it is, it still doesn't explain why
defragmenting 50 GB takes 35 hours.
> If you absolutely must use FAT32 (compatibility with other operating
> systems being the only plausible reason), the fastest way to defrag is to
> simply copy everything to another partition (or a spare drive), reformat,
> and then copy everything back.
Why can't defragmenters do exactly that when there's over half of free
space in the partition?
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Luckily this didn't cause any errors in the partition.
That's part of the reason. There's a sync before and after each block of
file movement, which makes for a tremendous amount of seeking. (Not each
allocated cluster, but each read/write pair. I don't know how big a buffer
it uses.) It also re-reads the free space bitmap for each file, so if you're
moving lots of small files to make room for a big one, that might take a
majority of the time right there. But it's all designed that you wont lose
data if someone else opens the file while it's defragging (read or write) or
if you pull the plug while it's going.
Plus, Windows defrags online, which means it has to keep the buffers up to
date in memory and things like that, not stomp on blocks just allocated to
another file, etc. I.e., you can be reading through a file while it's
defragging and not get errors from reading the wrong buffers and such.
Was this Vista or XP? Vista intentionally yields the disk up to other
processes while defragging. I also think Vista has a problem with a really
highly fragmented file. For example, I had a file with 300,000+ fragments
that Vista just choked trying to defrag. I fell back to sysinternals contig
(which works nicely and lets you do just individual files or directories
too). So it's not the kernel bit broken, but the UI/app on top of it.
> Why does it take such a humongous amount of time? I can't understand.
It *is* a lot faster if you have lots of room on the disk, IME. The Vista
defragger seems much slower than even the XP defragger.
> I also noticed that both defragmenters failed to use the free space
> in the partition (over 80 GB) to their advantage, and instead moving
> small clusters around one at a time. I'm pretty certain that if all the
> free available space was used to copy data, the whole process could be
> a lot faster.
That bit is in the kernel. You just say "move clusters X thru Y of this file
into the free space starting at Z" and it moves it. The likelihood that a
different version of the defragger will be significantly faster at
defragging a bunch of files is low, if the bottleneck is the I/O, because
they all use the same FCNTL calls.
http://technet.microsoft.com/en-us/sysinternals/bb897427.aspx
--
Darren New, San Diego CA, USA (PST)
There's no CD like OCD, there's no CD I knoooow!
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Fredrik Eriksson wrote:
> On Mon, 01 Jun 2009 22:07:26 +0200, Warp <war### [at] tagpovrayorg> wrote:
>> I have a secondary 180GB FAT32 partition
>
> That would be the biggest problem right there. FAT32 is really horrible
> on partitions that large.
Oh, I didn't even see that part. Yes, agreed - the free space map on a FAT32
partition that big is huge, and at the wrong end of the disk. :-) To defrag
a piece of file, it has to read the the directory entry, then the FAT to get
the location of the file, then read the FAT to find a free space, allocate
the free space, sync the FAT, copy the sectors, sync the sectors, change the
directory, sync the directory, then re-read the FAT and fix the previously
allocated sectors to be free, and sync the FAT. Lather rinse repeat.
--
Darren New, San Diego CA, USA (PST)
There's no CD like OCD, there's no CD I knoooow!
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Why would FAT32 become slow with partitions that large?
The FAT table is pre-allocated.
> And anyways, no matter how slow it is, it still doesn't explain why
> defragmenting 50 GB takes 35 hours.
Lots of seeks due to trying to make it reliable even when people like you
pull the plug in the middle. ;-)
> Why can't defragmenters do exactly that when there's over half of free
> space in the partition?
They can if they want. You're going to be even slower, tho, because you're
going to be seeking back and forth between the directory, the FAT, and the
data, and now you're seeking even farther.
--
Darren New, San Diego CA, USA (PST)
There's no CD like OCD, there's no CD I knoooow!
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> I can't understand why defragmentation takes such humongous amounts of
> time. There were something like 40 or 50 GB worth of fragmented data,
> but if you calculate how many times you can copy 50 GB inside a modern
> SATA disk, I think that in 35 hours you could copy it over and over
> thousands of times. (Assuming it would take 1 minute to copy 50 GB of
> data, you could copy it over 2000 times in 35 hours.)
>
> Why does it take such a humongous amount of time? I can't understand.
Defragging is limited primarily by how fast you can thrash the heads
back and forth, not by the maximum sequential transfer speed for the drive.
I bet solid-state drives would defrag a lot faster. (But then, why would
you bother defragging a solid-state drive in the first place?)
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
>> I have a secondary 180GB FAT32 partition
>
> That would be the biggest problem right there. FAT32 is really horrible
> on partitions that large.
I wasn't aware it *goes* that large.
I vaguely recall reading somewhere that to make FAT work on large
partitions, you make the cluster size larger or something weird like
that, which is probably not very efficient...
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Question: Assuming fully defragmented data, which is faster to read data
from a few big files: FAT32 or NTFS?
I tried goolging for this info, but couldn't find any definitive answer.
--
- Warp
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
| |
|
|
Warp wrote:
> Question: Assuming fully defragmented data, which is faster to read data
> from a few big files: FAT32 or NTFS?
If you're reading the entire file, I would guess it's roughly equal. If
you're seeking around a lot, I'd guess it's NTFS.
Why? FAT uses a linked list of clusters, with the first cluster pointed to
be the directory entry, as the allocated clusters of the file. (That's the
File Allocation Table: the linked lists.)
NTFS just stores the first cluster and number of clusters in the file table,
so calculating what cluster a given block of the file is on is probably
easier and requires fewer trips back to the start of the disk. I.e., you
don't have to read the disk to seek in the file.
You can even hear this if you have a floppy drive. Copy a 1 meg file to a
floppy, and you can hear the seek back to the FAT every handful of tracks
written.
This is just a guess, mind. I wouldn't think there's a whole lot of
difference on a machine with enough memory to hold the whole FAT in kernel
buffers.
--
Darren New, San Diego CA, USA (PST)
There's no CD like OCD, there's no CD I knoooow!
Post a reply to this message
|
|
| |
| |
|
|
|
|
| |
|
|
