http://groups.google.com/group/HAppS/browse_thread/thread/8663c04f3b104133

"Happstack scales, SQL fails!"

Um, yes, that's right. Some toy application that somebody wrote in their 
lunch break really is more performant than the likes of Oracle, 
Postgress, DB2 or SQL Server. Sounds completely plausible to me... Oh, 
wait. No it doesn't! :-P

I guess if you can afford to pay for 30 TB of RAM and you don't mind 
living with the absolute certainly that some day you *will* lose all 
your data and have no way of getting in back, then storing everything in 
RAM is perfectly OK. But it must surely be a vanishingly small number of 
scenarios where you have terabytes of data but you don't mind losing it all.

I especially love the subversive suggestion that data integrity is some 
optional feature that's nice to have but not really "necessary". As if 
you can somehow do without it. People whine about "it's too slow" as if 
it's just overhead that you somehow don't really "need".

Given the choice between a slow program that gives me the right answer, 
and some program that's much faster because it just spits gibberish at 
you and doesn't bother checking whether it even remotely makes sense, I 
know which one I'd choose. But maybe that's just me? :-P

Post a reply to this message

Invisible <voi### [at] devnull> wrote:
> Given the choice between a slow program that gives me the right answer,
> and some program that's much faster because it just spits gibberish at
> you and doesn't bother checking whether it even remotely makes sense, I
> know which one I'd choose. But maybe that's just me? :-P

does that mean you prefer an unbiased renderer to povray?

Post a reply to this message

>> Given the choice between a slow program that gives me the right answer,
>> and some program that's much faster because it just spits gibberish at
>> you and doesn't bother checking whether it even remotely makes sense, I
>> know which one I'd choose. But maybe that's just me? :-P
> 
> does that mean you prefer an unbiased renderer to povray?

Can't say I've ever tried one. Might be interesting to do though...

Still, the primary difference is that biased renderers usually manage to 
produce reasonable approximations to unbiased ones. Inconsistent 
databases give you complete gibberish, not an "approximation" to the 
correct answer.

Post a reply to this message

Invisible wrote:
> Um, yes, that's right. Some toy application that somebody wrote in their 
> lunch break really is more performant than the likes of Oracle, 
> Postgress, DB2 or SQL Server. Sounds completely plausible to me... Oh, 
> wait. No it doesn't! :-P

Yes. Let's compare it to AT&T's databases, which 30 years ago were on the 
order of 300 terabytes each of transactional database. I.e., they had 10x 
the transactional data as Facebook has. Seems to have scaled OK to me.

If you're willing to give up on transactional guarantees in order to save 
money on hardware and software, sure, go with an in-RAM database. Just make 
sure that (1) you don't process any actual financial data, (2) you don't 
process any data that actually leaves your system, and (3) you only ever 
implement one application that accesses the database.

What people seem to always miss in these designs is that their applications 
are enforcing their permissions and data integrity, rather than declarative 
statements describing them and having them enforced regardless of programmer 
bugs or maliciousness. Let's share the same database with three different 
companies/corporations and see how well that works out for you.

Let's imagine, if you will, that your application is (say) airline 
scheduling, where you have 100+ airports and 50+ airlines all flying 
thousands of planes, booking thousands of seats, and all *competing* with 
each other.

Or imagine a movie ticket application. Even if not competing as such, 
imagine the studios make available movie information, theaters publish times 
and customer reviews, places like Yahoo syndicate it, tickets are sold off 
the same database all over the country, you really don't want to be down for 
15 minutes when the thing *does* crash, the studios might not trust that the 
theater is reporting all the tickets sold, individual theaters might not 
want to upgrade their software regularly, etc.  Works with one giant 
key/value store with all consistency enforced by the application? Doubtful!

I'm certain Amazon does indeed use transactional relational databases for 
their services in *some* places. Catalogs? No - they don't change that fast 
and changes are idempotent. Charges to your credit card, and recording of 
shipping information? Yeah, I'm pretty sure that's transactional there.

> I especially love the subversive suggestion that data integrity is some 
> optional feature that's nice to have but not really "necessary". As if 
> you can somehow do without it. 

For some applications? Sure. Data integrity isn't that important. You're 
downloading today's movie listings and the download breaks? Try again. 
Anything with idempotent updates is a good candidate for scaling this way. 
Anything that a crash would cause the loss of the *need* for the data is 
too, such as "in progress" phone calls thru a phone switch.

One problem with these architectures is that when something *does* break, 
you have inconsistent data and no way to find it.

> But maybe that's just me? :-P

That's because you're not looking for venture capital.

-- 
Darren New, San Diego CA, USA (PST)
   Human nature dictates that toothpaste tubes spend
   much longer being almost empty than almost full.

Post a reply to this message