Orchid XP v8 wrote:
> Yes. But that's why you have a database - to manage the extreme 
> complexity for you, so you don't have to do it by hand. You seem to be 
> under the impression that this is somehow "not necessary".

I'm not sure why you think I said that. However, having said that, all you 
need is a decent file system. If you put each record in a different file 
with a unique name, you can do a fair amount of goodness with message queues 
and such without ever worrying about a database manager.

For example, the traditional mechanism for doing this is to create a file 
with a .tmp on the end of the name, and then rename it when it's ready to be 
processed. If it doesn't matter what order they're processed in, you take a 
sufficiently high-resolution clock, plus an IP address of the host, plus the 
PID of the process creating the file, and use that as the file name. You can 
have a bunch of machines updating the same file system without causing a 
problem.

The way CouchDB works it is when they commit work, they write the header 
twice, in widely spaced areas of the file, with a timestamp and checksum at 
the end of each. If one checksum is bad, they know the other is the right 
header. If both checksums are good, they know the one with the later 
timestamp is right.

Either of those work for writing messages in a queue, for example. Write the 
message as something like XML, where you can tell if it's complete, put a 
checksum at the end, and anything with a bad checksum when you start a 
recovery didn't get finished.

If you have separate log files that are append-only until they get too full, 
and you keep old copies of accounts, and those accounts are only updated 
relatively rarely (say, order fulfillment or credit card clearing), you can 
recover from failures by looking thru old logs when you come across a 
customer file that's corrupt, looking at the previous customer file, and 
replaying the transaction(s) since then.  That's basically what the database 
does when it crashes.

I'm just saying, it isn't that complex to accomplish "manually". You need an 
ACID database - it's just not that hard to build one if you can avoid making 
it general. The hard part of something like Oracle is not the ACID, but the 
ACID and high performance regardless of what you store on it. If you know 
what you're going to store and you code only the ACID properties you need 
(for example, hard-coded constraints instead of TRIGGER statements in SQL), 
it's pretty easy to do.

BTDTGTTS.

-- 
   Darren New, San Diego CA, USA (PST)
   My fortune cookie said, "You will soon be
   unable to read this, even at arm's length."

Post a reply to this message