Invisible wrote:
> On 09/03/2011 06:22 PM, Darren New wrote:
>> Having worked with Erlang, here's some comments.
> 
> In what capacity are you "working with Erlang"? 

I'm not, currently.

> While I doubt anyone actually *wants* unreliable software, there can't 
> be too many problems that require distributed programming.

Unless you want to listen to radio stations in several different cities and 
integrate the results. Altho I wound up writing my own stuff for that and 
not using Erlang, that was the experiment.

> Erlang is partially functional, in that functions are apparently 
> first-class, and it's got single-assignment and immutable data 
> structures. 

Functions in C are first class. That doesn't mean it's functional. The 
immutable data structures come from the single-assignment, except where the 
data structures aren't immutable, of which there are several.

> On the other hand, it doesn't enforce referential 
> transparency; you can put side-effects anywhere you like.

Well, that's the killer. To me, if every statement can have side-effects, 
then it's not a functional language. It's just single-assignment. You have 
none of the benefits and all of the draw-backs of a functional language in 
Erlang.

 > The document I
> read claims that it's considered beneficial to limit side-effects to 
> just a few places in the codebase, 

Except that message sending and message receiving are both side-effects as 
is spawning or linking another process, so that works pretty poorly. Plus, 
as I said, having the side-effects means you cannot (for example) use your 
own functions in guards. Essentially, nothing is actually functional in the 
language *except* assignment and a handful of built-ins. (Not even all the 
built-ins, for that matter.)

> the language itself doesn't /enforce/ any kind of strong separation 
> between pure and impure code.

For some reason, the Erlang enthusiasts think the crappy syntax and the 
single-assignment procedures are intentional and beneficial properties of 
the language. Like any niche language, the fanboi's refuse to accept that 
any compromise was made during its design.

>> If it was *actually* functional, you could put your own functions in
>> guards, for example.
> 
> Wait - you *can't* do this already?? o_O
> Why on earth would you design it this way?

Because your own functions aren't referentially transparent, because the 
language isn't actually functional, even at that level. The functions in a 
guard are the ones that *are* functional. Not even all the built-in 
functions can go in guards.

Actually, in the documentation, I think there's a note on each built in 
function that says whether you can use it in a guard or not.

>>> - The system is supposedly insanely reliable. People toss around "nine
>>> 9s up-time" as if this is some sort of experimentally verified *fact*.
>>
>> It is. They've been running a commercial switching network with it for
>> something like 10 years, and one of the six data centers lost power or
>> something and all the machines in it for 20 minutes once, so they had a
>> 15% downtime for 20 minutes out of ten years.
> 
> One data point is scientific fact?

It's a scientific fact that you can achieve nine 9's of uptime over the 
course of a decade with programs written in Erlang, yes.  Pretty sure that's 
really, really hard to do with any other programming language.

> (Presumably both processes have to be running within the same logical 
> "node", but then again also presumably you wouldn't have more than one 
> node on the same box except for test purposes anyway...)

You start up the run-time with a command-line argument that says "Hey, I'm a 
node in a distributed program". Wings3D doesn't do this, so it's not a node.

> As I say, this doesn't really seem to be spelled out anywhere else that 
> I could find, and it seems a rather crucial detail.

It's kind of in all the books and stuff. I think the PhD thesis is presumed 
required reading, if nothing else.

>>> less the same kinds of error handling that any other language provides
>>> - catch/throw, etc.
>>
>> Huh. I didn't even know it provided catch/throw. I've never seen that
>> used, actually.
> 
> Yeah they do. It's in there. Apparently the catch functions turn an 
> exception into a tuple begining with 'EXIT' and followed by some stuff. 
> Using throw you can make catch return an arbitrary data structure. 

Ah, I see. Yes, I had forgotten about that.

> (Quite how you tell the difference between an exception and a normal 
> result is left to the imagination... 

Well, an exception will start with EXIT, while a regular result is unlikely 
to, unless you specifically confuse things that way.

> It looks like you can use throw to 
> make it look like the catch block exited normally.)

Yes. Basically, it lets you return early.

> Right. So it just sends heartbeat messages? (I presume the bit about 
> rebooting the machine is optional; you wouldn't want your PC to reboot 
> just because Wings 3D crashed, for example.)

Yes, it's a separate monitor process. It sends heartbeats as well as watches 
for a couple other simple things.

>>> Message sending has the curious property that message delivery is
>>> /not/ guaranteed, but message ordering /is/ guaranteed. Like, WTF?
>>
>> Message delivery isn't guaranteed because it's asynchronous, and the
>> remote machine may crash between the time you send the message and the
>> time the remote machine receives it. Ordering *is* guaranteed because if
>> the remote machine crashes before it gets message #5, it won't get
>> message #6 either.
> 
> You mean that if message #5 doesn't arrive, message #6 is actually 
> guaranteed not to arrive?

If message #5 didn't arrive because the receiving machine crashed, yes, 
message #6 isn't going to be received either. If message #5 didn't arrive 
because the network switch died, then #6 isn't going to be delivered either.

> TCP guarantees both delivery and ordering. Unless the connection 
> actually breaks. So is that all Erlang is saying? That it if one of the 
> machines catches fire, Erlang can't guarantee that messages will be 
> delivered to it?

Yes, exactly.  More precisely, you can't tell if the message was received 
and processed or not.

> These people are /serious/ about reliability. o_O

Well, yes, that's how you get 5 minutes of downtime over the course of 10 years.

>> Knowing the receiver got the message is as simple as adding code to send
>> an ack each time you handle a message.
> 
> Network protocols 101: It's not that simple. It is possible for the 
> message to arrive, but the acknowledgement itself to fail to be delivered.

If the ack fails to be delivered, it's because the receiving machine (nee 
sender) has crashed and hence doesn't need to see the ack.

Well, not quite. If the network between the machines dies, each machine is 
going to think the other crashed. This makes a mess. Mnesia has all kinds of 
crap in place to recover from this, especially given that you might make 
independent updates to the same table in two places.

> Now, if you're running on TCP, so that ordering and delivery are 
> guaranteed (unless there's no transport or no endpoint), the only way 
> this can happen is if the transport or the endpoint dies after the 
> message is received, which is probably just as bad as the message not 
> being delivered...

Right. The point of their warning is, you have to deal with messages not 
making it one direction or the other. You don't have to deal with the 
messages being out of order. You have to deal with the fact that if you tell 
a remote machine to do something, it might not happen. You don't have to 
deal with the fact that if you tell a remote machine to do two things, it 
might do them in a different order.

> Thing is, I can open the REPL and type in a function, and then ask to 
> spawn that on a remote node. How does *that* work?!

Hmmm. Actually, I think perhaps it *is* willing to ship functions around. 
The details escape me at the moment. Maybe you can send functions but not 
modules? I don't honestly remember.

> I'll bet. No doubt somebody has written libraries for other languages to 
> speak the same wire protocol.

I don't know. Probably, but I'm not sure why you would.

> I notice that connecting one Erlang node to another causes the two nodes 
> to connect to *all* the nodes that each endpoint knows about. 

Yes, unless you tell it not to.

> I'm not actually sure why though. 

Reliability. ;-)

> You would think that a fully interconnected 
> mesh would quickly exhaust the supply of TCP ports too.

Unless you have more than 50,000 machines, unlikely.

>>> In other words, you're authorised if you can
>>> actually access the system in the first place.
>>
>> Yes, exactly.
> 
> Ouch.

One of the OTP functions shows how to authenticate and encrypt the 
connection en Erlang code, IIRC. That's also one of the tutorials.

Imagine a room full of a few thousand processors. One outgoing internet 
connection running thru a firewall that's happy to VPN anything going out to 
a known location, and which drops anything going in or out from an unknown 
IP.  Why encrypt everything?

Actually, I had forgotten. There is a "cookie" you can set, and anyone 
connecting needs the same "cookie". So there is a minimal security level, 
not unlike the similar feature in X servers.

> You say "sure", but the documentation sings about how reliable Erlang is 
> and makes it sound like all this stuff just happens by magic, because 
> Erlang is so wonderful. The truth is actually that Erlang just 
> implements the minimal features required to let _you_ build complex, 
> robust, distributed applications.

What, you're believing fanboi's?

If you want the version that actually *does* magically distribute and make 
reliable your code, you want to read about Hermes, a language designed by Strom.

>> The two-versions limit is because the code itself isn't GCed. They
>> document that they could have GCed the code, but that would have added
>> lots of overhead for something relatively rare.
> 
> Fair enough.

Plus, you should be dealing with the occasional "process suddenly ends for 
no reason" exception robustly anyway. :-)

> You'd have to be able to do a lot of things. It would require careful 
> application design, but it doesn't look especially difficult. (To get 
> the framework done, anyway. Making the application itself sounds really 
> hard, in Java or Erlang.)

Heck. Honestly, you can do it in Ada.  Ada has facilities for declaring some 
library code as capable of being hot loaded and unloaded.

> I don't mind which level the assistance is implemented at, just so long 
> as there *is* some. ;-)

I found it annoying that there's no global roadmap. So you read about 
something and it says "If X happens, a message is sent to the error logger." 
  WTF is the error logger? Where's the documentation. Then you track that 
down, and it's all talking about bits and pieces that are also not 
documented, etc, with the assumption that you have a bunch of people who 
wrote the error logger hanging around to tell you how to find where the 
errors got logged and such.

>> Realize that most of these "systems" you're talking about are actually
>> implemented as Erlang code.
> 
> Hey, it worked for Smalltalk. ;-)

Yeah, basically the same idea, except instead of an "image" you actually 
have a bunch of running processes.

>> I agree about the syntax etc. It's quite possible a better type system
>> would make it easier.
> 
> Escaped experiment, eh? ;-)

That's exactly what it was. Except that by the time it escaped, there were 
millions of lines of production code written in it, so you couldn't even 
reasonably fix it.

>> You should also check out Hermes, which is essentially the same thing
>> with an astoundingly strict and high-level type system.
> 
> Heh, there is never enough free time. ;-)

I don't think the book is available any more, either. I should scan mine.

> The advice presumably being that workers should only receive "real" 
> messages, and supervisors should only receive exceptions. Then there can 
> be no ambiguity. (Until the supervisor's supervisor asks it to do 
> something...)

Well, more like, normal worker processes shouldn't communicate with messages 
that start with the atom "EXIT". It's not that hard. It's like saying "don't 
make closing the TCP connection the confirmation."  There's only one or two 
patterns that the error messages use. You shouldn't use "Close" as the name 
of the routine to invoke when you want your object to keep running, either.

> Actually, that's a point. Each process has *one* mailbox. Not one 
> mailbox for each process it's communicating with. Just *one*. So you 
> have to manually figure out who each message came from - usually by 
> having the sender put a reply address in manually.

Yep.

Hermes gives you as many mailboxes as you want, with as many connections to 
each mailbox as you want, and you can send both connections and mailboxes in 
messages to other processes. So if you want hot-swap code, write your 
processes as being capable of packaging up its mailboxes and sending them to 
a new process.

> So you're saying that all the "interesting" stuff like the SSL 
> implementation is actually just an off-the-shelf C library?

It's possible.

> I never did understand what's so great about Tk. It looks horrid, it's 
> really hard to use, and it's almost impossible to get the layout you 
> want...

Well, no, it's really easy to use, it's easy to get a good-looking layout, 
and it only looks horrid because it looked cutting-edge 10 years ago and 
nobody ever improved the look of it since then. (Altho they've worked on 
that more lately.)

That's exactly why half a dozen non-Tcl languages actually went so far as to 
embed a Tcl interpreter into their code in order to be able to use Tk. I'm 
pretty sure it's not Tk that's at fault here. ;-)

>> In general, network switching isn't that hard real-time, because you
>> usually have custom hardware to do the switching. A 5ESS (which can
>> handle up to 800,000 phone lines) runs on two 6800s, one of which is a
>> hot spare. But of course there's a large lump of hardware that's routing
>> the individual bytes here and there, so the 6800 only actually gets
>> involved when you connect or disconnect, basically.
> 
> Wait, a 6800? As in, the chip *before* the obsolete 68000?

Yes.

> Damn. If the actual switching is in hardware, that the heck do you need 
> the CPU for at all?

For deciding what to connect. You pick up the phone? The 6800 connects your 
phone to the dial-tone generator. It's not sitting there generating the 
waveform in software. You push a button? The 6800 routes your line to the 
DTMF decoder, which signals the 6800 each time you let go of the tone. When 
you've dialed enough numbers, the 6800 figures out what line you're supposed 
to connect to and plugs you through.

> Actually, one of the reasons I wanted to learn about Erlang was to see 
> if I can design something that does the same thing, but isn't insane.

Check out Hermes first. :-)

> Well, yeah, I meant for pattern matching against a value. Receiving 
> messages as a bit special.

Right.

> OK. So that's why the state is explicitly threaded (quite apart from 
> Erlang apparently having no other way to handle running state). That 
> still doesn't explain why I can't put several related behaviours into a 
> single module. Perhaps I *want* to replace them all at the same time.

I don't know. Go work for Erricson. ;-)

>>> Many things about Erlang still remain unexplained. I still have
>>> absolutely no clue WTF an "atom" is.
>>
>> An atom is an interned string. Same as a "symbol" in LISP or Smalltalk.
>> The trick is that in a message, an atom is an index into the atom table,
>> so it's short regardless of how long the atom is.
> 
> I never really understood what an interned string is. (And hence what 
> the difference between a Smalltalk string and a symbol is.)

I explained this for Warp a couple weeks ago. An interned value is a 
reference value where reference equality equals value equality.

Or, in normal speak, an interned string is one where there's only one copy 
of any given string, so you can compare the addresses where two strings are 
stored and tell if they're equal.

In Erlang, you can think of an atom as one element of a giant global enum of 
all the atoms you've ever used in your program. It's nothing more than an 
index into a table that happens to be readable.

> So you're saying that the compiler replaces all occurrences of an atom 
> with a pointer to a single instance of the string? (And hence, you can 
> compare atoms for equity by simple pointer equity rather than string 
> comparison.)

Basically, that's what an interned string is. An "atom" is implemented as an 
interned string, but it's syntactically different. It's ... an atom. It's a 
string treated as a single atomic value. You can't ask for the third 
character of an atom, for example.

>>> Nor have I discovered what this "OTP" thing that gets mentioned every
>>> 3 sentences is.
>>
>> It's the libraries that take the basic operations Erlang supplies (like
>> "load new code" or "spawn link") and turn it into things like
>> installable packages.
> 
> Right, OK. So what does it mean when they say "the compiler is an OTP 
> application"?

It means the compiler is a function in the OTP libraries that you invoke in 
order to compile the code.

> OK, so, I can register a PID with a particular name. Is that name local 
> to just that node? Does every node in the system know about it? How do 
> you avoid registering a name that some unrelated application has already 
> registered? If I ask for a particular name, how does it figure out which 
> node that's on? If a node goes down, does the name get taken out of 
> service? How do you handle the same service being available from several 
> nodes? (For load-balancing or fault tolerance, perhaps.)

I don't remember all this stuff. Registering as a known name passes your 
mailbox and the name to a process that proceeds to distribute it and answer 
queries about it. You can look at the documentation for it to see what it does.

>> It's basically a distributed transactional system on top of the built-in
>> database tables whose TLA name escapes me at the moment.
> 
> Right.

ETS is the name of the sub-system functions.

>> Note that all these things, OTP and Mnesia, along with stuff like the
>> compiler, the debugger, the profiler, the package manager, etc etc etc,
>> are all documented as "here's the functions you use to do it in Erlang."
>> Almost none of these things are actually command-line tools.
> 
> Nothing wrong with that. The question is how much detail the 
> documentation explains. ;-)

The other problem is that all the documentation assumes you've read the 
previous documentation and knows what they're talking about, but it doesn't 
actually give you pointers to any of the previous documentation.

So if you want to use mnesia, you read mnesia, but it assumes you know what 
ets and gen_server are and etc. So mnesia will have a function and say "this 
only works on ram_raw tables" or "that function starts a new server" and 
assume that you know "ram_raw" is something that ets implements (not Mnesia) 
and that a 'server' in this context specifically means a gen_server module.

You'd like mnesia, tho. It's a lot like STM in some ways.

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message