Invisible wrote:
> Darren New wrote:
>> http://queue.acm.org/detail.cfm?id=1394128
>>
>> Does a good job of describing how to go about breaking ACID for 
>> eventual consistency and higher availability.
> 
> It seems to me that "BASE" fundamentally requires an ACID-compliant 
> database.

They describe it in terms of that, but no, it doesn't. Because it's the "C" 
they're leaving out, you see. They're saying "this is how you build an 
unboundedly large BASE database from smaller ACID databases that may or may 
not be online at any given time."

> They're only talking about relaxing consistancy requirements 
> in a few key areas. A completely non-ACID database wouldn't function 
> properly. (E.g., if the database engine's own metadata became 
> inconsistent, the database engine wouldn't be able to read the database 
> any more.)

Certainly. But that's not what the "C" means in ACID. Lots of people claim 
that's what the "C" means, because they don't have the right kind of "C" and 
want to claim they do. "I don't corrupt the entire database when the server 
crashes" isn't something to brag about, any more than "I don't stall out 
driving down the freeway" is a selling point for an automobile.

> All I can say is, I hope *I* never have to deal with a database so vast 
> that I have to risk having it give me gibberish from time to time just 
> so that it's "fast enough".

It's pretty normal, actually. For example, in the credit card world, "real 
time" means "we do it every day, instead of waiting for the end of the 
billing cycle."  Do you think when the merchant gives you a refund, you can 
then go and spend that money at the next store over 5 minutes later?

> "For constraints to be applied, the tables must reside on a single 
> database server, precluding horizontal scaling as transaction rates grow."
> 
> Um... why?

A constraint is the "C" kind of consistency. It says you can't update one 
table without checking another table it's OK. (Foreign Keys are one kind of 
constraint, but there are lots of others.) If the table with the primary key 
and the table with the foreign key are on different servers, and the table 
with the primary key crashes, you can't check when you insert a row into the 
other table that the constraint is still good.

> Still, now Amazon's message service (whatever the hell it's called) 
> makes sense.

Uh, sorta. More like Microsoft's queue mechanism, which is a lot like 
Amazon's basic mechanism, except the queue messages can interact with the 
same transactions that your SQL server and file system are interacting with. 
You can do things like create a transaction, update some SQL rows, queue a 
couple messages, and rename a couple of files, then decide you want to roll 
it all back, and everyone involved will never see the SQL rows, the 
messages, or the file name changes.

Amazon's message service is for doing that sort of thing, but it takes a 
different approach to the reliability - you peek the message off the front, 
which locks it, and then it's a timeout that releases the message. It's 
actually pretty poor if you want to do something like queue POV-Ray renders 
for which you have no idea how long they'll take.

> I do feel, though, that the mechanisms described for processing work now 
> and restoring consistency later by passing messages, and detecting when 
> consistency has returned and so forth really ought to be part of the 
> database engine rather than application code...

It is. This is how it does things when you have *multiple* database engines, 
and you want to keep going even when the hardware for some of it has burst 
into flames.

Lots of systems are built like this internally, too. For example, there are 
mail systems where each rule on processing the email[1] winds up pulling the 
email off the queue and putting the new one on the end. Same sort of thing.

[1] Like, "if it doesn't have a domain on the from address, put this domain 
on it before it leaves the local intranet."

-- 
   Darren New, San Diego CA, USA (PST)
   My fortune cookie said, "You will soon be
   unable to read this, even at arm's length."

Post a reply to this message