On 5/13/2011 10:11, Orchid XP v8 wrote:
> I'm fairly sure I posted one in the other group a while back.

Yeah, I was just wondering if you had something convenient. I was curious 
waht it looked like.

> Actually calling a C function is trivial. Passing a Haskell function pointer
> to C is trivial. What is very *non*-trivial is getting data marshalling and
> memory management to work.

Well, yeah, that's why I was curious. :-)

> I'm sure lots of people are doing this. After all, the basic abstractions
> are not complex.

They actually are pretty complex if you actually want it to be reliable. 
Think this: You spawn a process, but before it starts up, it dies. Do you 
handle it?  Lots of edge cases to get right.

> Perhaps. Perhaps the data is already in some distributed database, so it's
> already persistent regardless of whether a node dies or not. (Isn't the
> system supposed to recover from node death anyway?)

Yes.

To give an example, mnesia (erlang's db) works by storing all the records in 
a tree, with indexes being other trees. When you update a tree, it appends a 
record to a data file saying what you changed - i.e., it logs the change. If 
your node fails and you restart it, it replays the log to recreate the 
database. Every once in a while, it takes the whole database and writes it 
out to a new file, and starts a new log.

So restarting a node requires first reloading all the records, one at a 
time, into a balanced tree. Then replaying all the changes since the last 
time the node was saved out to disk. Then playing all the changes made since 
the node went down, as copied from some other node.

On a gigabyte CSV file of a half dozen fields per line, on a 3GHz machine 
with 100MBps data transfer, this takes about an hour for the first step of 
just reloading the saved file. Say you have 50 machines to upgrade. You're 
now talking 2 wall days just to restart the new processors, ignoring however 
long it might take to dump the stuff out.

> With hot update, you *still* have to manually write a bunch of code to do
> any data conversions if data structures have changed, or do invariant checks
> if the data's invariants have changed. Either way, you gotta write code.

Sure. But you don't have to write code to cleanly shut down *other* 
processes when some unrelated process on the same machine gets changed. If I 
want to upgrade firefox, I don't have to write any code in the firefox 
installer to save out my current Word document, close Word, update firefox, 
reopen Word, and reload my Word document.

You can also convert things slowly, using the old data in the new code until 
all the old code is gone, then slowly work thru the data (perhaps updating 
it each time it's touched) to the new format.

> Yeah. As I say, the current system is entirely predicated around everybody
> having the same code. As soon as that's not true, it's game over.

That, I think, is the fundamental problem with systems like Haskell, and the 
fundamental reason why dynamic typing may very well win in this situation.

> It's already dealing with receiving messages of arbitrary type even in the
> presence of static typing. All that's needed is some way to compare types
> beyond "do they have the same name?"

Oh, OK.

> Yeah, passing channels around looks quite useful. It just seems a pitty to
> have more or less duplicated functionality in two incompatible systems -
> mailboxes and channels. Couldn't they have implemented a mailbox as a
> channel of arbitrary type or something?

Two people designing the two systems, maybe?

> Well, OK, let me rephrase: There is no situation where an STM program gets
> so stuck that nothing can make progress. The worst that can happen is that
> one particular transaction gets stuck. (And they're talking about adding an
> API to detect / do something about this if it actually happens.)

Sure. Linux, way back when, would do the same thing with file read/write 
locks. If you had four read transactions lasting 3 seconds, with a 1 second 
gap, nd you had them all staggered at 1 second intervals, a write lock would 
never take. A hold-over from the original UNIX design where kernel calls got 
woken up just based on signaling some address they're waiting on and letting 
the processes fight it out.  That did get fixed a couple years later, tho.

The problem is that when it happens, it kills you. :-)

-- 
Darren New, San Diego CA, USA (PST)
   "Coding without comments is like
    driving without turn signals."

Post a reply to this message