>> holding file data will still be intact. (Formatting with, say, ext2 
>> takes a tiny fraction of the time that FAT or NTFS formatting takes,
> 
> D'oh?  Hardly.   I've never had an NTFS format take more than a half a 
> minute or so, and formatting a 750G drive with ext3 on the same machine 
> takes 10 or 15 minutes.

For me, it typically takes about 20 minutes to do a full NTFS format on 
a large partition. (Doesn't seem to matter how big the drive is - 
presumably because larger drives are also newer and hence faster.)

> Ext2/3 stores i-nodes all over the disk, preformatting them. This takes 
> time when you have a couple gig of i-nodes to fill out.

For me, ext2 takes about 15 seconds or so - very much faster. Comparable 
to an NTFS "quick format".

>> I am unsure as to whether #4 and #5 are different in any way. Both 
>> seem to take the same amount of time...
> 
> On my drives, at least one of the maxtor "format back to factory-fresh" 
> only wrote the first sector on each track or something. It finished way 
> too fast to be writing the whole drive. Unless the drive had a command 
> built in that wiped the entire track in one rotation or something, 
> rather than actually having to transfer the data from memory to the 
> drive for the whole thing.

On the Maxtor tool I have, there are two options: "low-level format 
(quick)" and "low-level format (full)". The former erases the beginning 
and end of the drive. The latter erases everything. (And takes a 
ridiculous amount of time to complete.)

Clearly I'm going to have to actually do some experiments. :-P

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Jim Henderson wrote:

> First rule in data security:  Never assume any data is unrecoverable.
> 
> Second rule in data security:  Never assume nobody is watching.

Third rule in data security: There's no point in overkill. :-P

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

>> I see lots of talk of *boken* hard drives, but not much about
>> deliberately erased media.
> 
> They've got the equipment for that.  I used to talk with them 
> occasionally, and they did do forensic analysis for law enforcement back 
> then.  Don't know if they still do.

Forensic analysis isn't too hard if the person didn't get a change to 
destroy their data before the PC was ceased. ;-)

>>> Or use something like this: http://freshmeat.net/projects/mobiusft/
>> I can't even find any documentation explaining what this *is*...
> 
> It's a forensic data toolkit.  Used for investigating criminal activity 
> that's been stored on hard drives and then wiped/erased/whatever.  Often 
> times, you don't recover a file, but you recover data on individual 
> blocks (think like chkdsk).

Right. But if I've erased every individual block on the drive (rather 
than just, say, reinstalling Windows on it) then it won't find anything.

>>> - people have been convicted using evidence recovered from a drive
>>> without using "a microscope" (and BTW, how would that help?  The data
>>> is encoded in a magnetic field, a microscope won't see that).
>> I didn't mean a light microscope - I was actually thinking of a Magnetic
>> Force Microscope...
> 
> Well, all I know is that I know people who actually *do* this kind of 
> work on a regular basis.
> 
> Doing a DoD style wipe is generally sufficient, but as others pointed 
> out, the point from a data security standpoint is to make the cost of 
> recovery more than the value of the data when recovered.  In *most* 
> cases, a wipe is sufficient, but it really depends on how valuable the 
> data is to your competition.

1. These drives probably never contained any live data in the first 
place. ("Live data" as in data generated in our lab.)

2. Any such data they did contain is going to consist of a huge chunk of 
numbers, with little or nothing to identify what the hell they relate to.

3. Any such data will be at least 5 years old.

Really, it would be far, *far* easier to just have somebody come work 
for us and secretly spy on our activities... Much cheaper than James 
Bond data recovery techniques.

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Gail Shaw wrote:
> One the subject of format, what does Low-level format actually do? 

Nowadays, it means many different things, depending on the drive.

On old floppies, you used to have to do a low-level format by writing a 
pattern of bits to the track as it spun. Basically, since floppies don't 
spin exactly the same speed all the time, you need to first write a lead 
in, then a bunch of sectors, then enough of a lead-out that you are sure 
you overwrote the lead-in without overwriting the first sector. Each 
sector consisted of a set of bits that said "here comes a sector", then 
the track and sector number and sector length, then some mungable bits 
that gave the read/write head time to switch from reading to writing, 
then bytes that said "here comes the sector data", then the bytes of the 
sector, then the flag that said "here comes the end of the sector". The 
byte pattern for "the body of the sector" when you formatted it was 
traditionally E5 (hex), which is why when you delete files on FAT (or 
CP/M) file systems, it overwrites the first byte of the directory entry 
with E5.

So that was the low-level format - taking a completely erased disk and 
writing out all the header information that said which bytes were 
included in which sectors.

(The Amiga stored more on the disks by basically rewriting the entire 
track each time the sectors were written. Just as fast, and you didn't 
need to be switching between read and write every time.)

Note that floppies are actually serial devices. So to store more than (I 
think) three or four bits of 1's or 0's in a row, you have to inject 
differences between. (So you have to encode 00001111 as 00101101, or 
something. MFM is one such common encoding.) Kind of the same reason a 
slow modem has clean tones and anything over 2400 has a "noisy" tone - 
it's doing the same sort of randomized bit stuffing. The Amiga DMA 
hardware could do that sort of stuffing without CPU intervention, so it 
wasn't a lot of overhead to rewrite an entire track when you changed one 
sector.

Nowadays, floppies come pre-low-level-formatted. I think if you put in a 
"raw" floppy (i.e., one with a file system the OS doesn't recognise), it 
will do the low-level format. Otherwise, you couldn't erase a floppy 
with a magnet and then have it work.

With a hard-drive, it used to be the same sort of thing. Some hard 
drives, however, now have one platter dedicated to nothing but 
synchronization information, basically. So all the information about bit 
timing and such is on one platter. In other words, instead of a serial 
stream, there's a parallel stream. Doing a "low level format" means 
rewriting that platter, which is much more sensitive to problems of 
rotational speed and such. In other words, they do they low-level format 
on a machine with much higher precision than the motor they put in the 
drive they sell to you, so it's very hard to recreate the patterns.

Nowadays, a "low-level format" on a hard drive means to write some 
constant pattern over all the bytes you can write to.

-- 
Darren New / San Diego, CA, USA (PST)
  Helpful housekeeping hints:
   Check your feather pillows for holes
    before putting them in the washing machine.

Post a reply to this message

Orchid XP v8 wrote:
> For me, it typically takes about 20 minutes to do a full NTFS format on 
> a large partition. (Doesn't seem to matter how big the drive is - 
> presumably because larger drives are also newer and hence faster.)

Oh, are you doing a quick format, or a full format? A "full format" 
actually does read the entire drive. Ext apparently won't do a "full 
format". You have to do a quick format, then scan the disk for errors.

>> Ext2/3 stores i-nodes all over the disk, preformatting them. This 
>> takes time when you have a couple gig of i-nodes to fill out.
> 
> For me, ext2 takes about 15 seconds or so - very much faster. Comparable 
> to an NTFS "quick format".

Try doing a "quick format" with NTFS. The difference between a 15-second 
ext format and a 20-minute NTFS format gives me the impression that your 
NTFS format is actually looking for flawed sectors to map out.

> Clearly I'm going to have to actually do some experiments. :-P

Either cipher /w:x: on Windows (or at least Windows recent enough to 
actually have that functionality) or dd if=/dev/urandom should do the 
trick.

I'll try formatting a spare partition I have here with NTFS and ext2 and 
see what I get in timings.

-- 
Darren New / San Diego, CA, USA (PST)
  Helpful housekeeping hints:
   Check your feather pillows for holes
    before putting them in the washing machine.

Post a reply to this message

Doctor John wrote:
> 
> GPG shreds files on all my machines. It can also encrypt a whole
> filesystem. What else do you need?
> 

Deniability and perfect forward secrecy?

Post a reply to this message

Doctor John wrote:
> m_a_r_c wrote:
>> good quality coke
> 
> Snort! :-D
> 
> John
> 

Yeah, why burn it when that's still an option?

Post a reply to this message

Darren New wrote:
> I'll try formatting a spare partition I have here with NTFS and ext2 and 
> see what I get in timings.

OK, on a 40 gig partition,
ntfs /q - 5 seconds
ntfs full scan - about 20 minutes. (I stopped 10% in)

ext2 - 23 seconds
ext2 -c (full scan) - about 20 minutes.

ext3 - 27 seconds.
(Note the extra 4 seconds is roughly a constant, independent of disk size.)

An ext3 format of a 750G drive at work took something like six to ten 
minutes, but I might have had other layers in there (raid or encryption 
or something) that I don't recall.  I do recall doing the equivalent of 
chkdsk /f /r on the ext3 drive went about 36 hours for the first pass, 
and then said "starting second pass" without even any indication of how 
many passes it was planning to do, so I killed it at that point. :-)

-- 
Darren New / San Diego, CA, USA (PST)
  Helpful housekeeping hints:
   Check your feather pillows for holes
    before putting them in the washing machine.

Post a reply to this message

Invisible wrote:
> As far as I know, both option #5 and option #6 make it impossible to 
> read any useful data from the drive just by plugging it in and asking 
> the drive electronics to read the data off the disk surface. In either 
> case, you would have to physically dismantle the drive and crawl over it 
> with a microscope - which is damned expensive. Using random data (and 
> multiple passes of it) reputedly makes this process harder - but not 
> impossible. Non-sequential writes are supposed to make it harder still.

The US military overwrites (with pseudorandom data) five times before a 
classified drive becomes unclassified.

I suppose a drive could be designed so that a single pass is good 
enough, but if you're in that big a hurry, thermite is your friend.

Regards,
John

Post a reply to this message

Darren New wrote:

> Oh, are you doing a quick format, or a full format? A "full format" 
> actually does read the entire drive. Ext apparently won't do a "full 
> format". You have to do a quick format, then scan the disk for errors.

Full format.

Does it just scan the disk, or does it actually write data to it? I was 
under the impression it wipes the cluster descriptors, yet doesn't 
actually erase every single block on the disk...

>> For me, ext2 takes about 15 seconds or so - very much faster. 
>> Comparable to an NTFS "quick format".
> 
> Try doing a "quick format" with NTFS. The difference between a 15-second 
> ext format and a 20-minute NTFS format gives me the impression that your 
> NTFS format is actually looking for flawed sectors to map out.

An NTFS quick format takes about 15-20 seconds, same as ext2.

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message