OK, so Erlang is something of an interesting case. There are several 
reasons for this:

- There are many functional programming languages. (Haskell, OCaml, 
Clean, and for some reason people keep calling Lisp "functional" too.) 
However, Erlang is *the only* one that could be considered "commercially 
successful", as far as I can tell.

- Other languages attempt to be *concurrent*, but only Erlang is 
*distributed*.

- The system is supposedly insanely reliable. People toss around "nine 
9s up-time" as if this is some sort of experimentally verified *fact*.

- The language claims to do all sorts of wacky, far-out stuff like 
trivial concurrency and distribution, hot-swapping running code, 
detecting and correcting run-time errors, and so forth. I want to see 
how it's done.

I had a look at Erlang.org, but it's difficult to find anything that 
really explains the interesting parts of the system. I can find plenty 
of "Hello World" programs, and "this is how you run stuff remotely", but 
nothing about "this is why we did it this way", or "this is how it 
works" or "this is how to use it to build non-trivial applications".

Suffice it to say, from what little I could discover, I didn't like what 
I was seeing. Like most commercially successful languages, Erlang is 
obtuse, complex, ugly and kludgy. Much like C, Java or anything else 
wildly popular. It's abundantly clear that Erlang is about as 
"functional" as Lisp (i.e., not at all). Still, people claim that 
Ericsson's entire business is based on it, and lots of people are using 
it, so it must have got *something* right.

Eventually I discovered a file entitled Armstrong_theses_2003.pdf So I 
spent the last two days reading this 300-page tome in the hope of 
enlightenment.

Predictably, the author spends most of that page count either repeating 
himself, or telling me in minute detail about things which are of 
utterly no interest to me. However, I did learn some interesting things.



In a way, Erlang is a bit like Haskell: Fundamentally, it does something 
extremely inefficient. And yet it seems to work just fine.
In Haskell's case, it's not allowing you to modify things in-place 
(except under very controlled conditions). In Erlang's case, it's 
refusing to allow processes to share state. This immediately implies 
that if you want to send data from place to place, you must copy it all. 
It's rather like Haskell's restriction, only much worse.

Then again, if one process is operating on a completely different node, 
you are *forced* to copy data from place to place anyway. It is 
unavoidable. So in a way, forcing you to do it all the time just makes 
it that little bit simpler to move from local to distributed coding.

One thing that rapidly becomes clear (and doesn't seem to be mentioned 
anywhere else) is that for Erlang, processes are about more than just 
concurrency or distribution. They are about fault isolation. A 
fundamental design assumption of the Erlang system is that stuff /will/ 
break, and /when/ it does, we want to maximally limit the damage. Part 
of the philosophy is that if one process dies, it shouldn't hurt anybody 
else.

Whereas in some other system you might structure things as separate 
processes for increased performance or because it's more logical to 
program it that way, in Erlang you might do this for no other reason 
than the fact that one task can sensibly continue if the other fails. 
You might use processes purely for error-handling reasons.

Another thing that becomes painfully obvious is that writing your 
program in Erlang does not somehow magically make it reliable. The 
language doesn't implement any kind of automatic error recovery system, 
as you might be mislead to believe. Instead, Erlang provides more or 
less the same kinds of error handling that any other language provides - 
catch/throw, etc. You can nest exception handlers, re-throw or modify 
exceptions, generate synthetic exceptions, and so on. And, as usual, any 
uncaught exceptions at the "top level" just make the process die.

The difference is what happens when a process dies. Rather than just 
dying, this situation is *detected*, and you can *react* to this. You 
can make it so that related processes get auto-killed as well 
(presumably because they can't sensibly complete without the dead 
process), or you can make it send a notification to some monitor 
process. The *language* itself allows notifications to be sent, but 
nothing more. The *libraries* allow you to do sophisticated error 
recovery, but *you* have to implement this. It doesn't happen by magic, 
as many seem to suggest.

Very importantly, the system correctly handles things other than 
software exceptions. Detecting division by zero is one thing. Detecting 
that the machine in Australia that you were talking to just got hit by a 
small pyroclastic flow is another. Obviously, no documentation actually 
describes how this works.



So how does Erlang actually work then? I mean, you can create processes. 
OK, then what? Well, as best as I can determine, each process has a 
"mailbox", and you can send asynchronous messages to any process that 
you have the address of. (And these messages can contain process 
addresses.) This works the same way both locally and remotely.

Sending is a non-blocking operation. Receiving is a blocking operation, 
and it makes use of Haskell-style pattern matching (which seems like a 
rather neat fit here). You can also time out waiting for a message - 
something which would be really hard to implement yourself, and which is 
probably extremely important in networking code.

Message sending has the curious property that message delivery is /not/ 
guaranteed, but message ordering /is/ guaranteed. Like, WTF? I'm not 
even sure how it's possible to tell that the messages are in order 
without being able to tell if you got all of them, but whatever. Seems 
very, very random to me. The thesis claims that implementing delivery 
checks yourself is easy, but implementing ordering checks is very hard. 
So they put the hard thing into the language, and left the easy thing up 
to you if you need it. Um, OK?

One of the very trippy things about Erlang is that I can be sitting on 
my PC in England, and I can just tell some Solaris server in Brazil to 
spawn a new process. As expected, no word on how the hell this actually 
works. I would have *thought* this means that the necessary code gets 
beamed over the wire, but some documentation seems to suggest that you 
have to install it yourself. And yet, you can apparently send arbitrary 
functions as messages, so...?

There's also no word on authentication. I mean, how does the machine on 
Brazil know that I'm authorised to be executing arbitrary code on it? My 
best guess is that if you're a telecoms giant, you have a private data 
network that nobody else can physically access, and you use that to 
control your equipment. In other words, you're authorised if you can 
actually access the system in the first place.



It's a similar story when we come to code hot-swapping. Surprise, 
surprise, it turns out that you can't just magically hot-swap code, just 
because it's written in Erlang. No, you have to do *actual work* to make 
this possible.

The language itself allows two (and only two) copies of the same code to 
be loaded at once, an "old" version and a "new" version. And it provides 
a way to switch from one to the other for a specific process. And that's 
*all* it gives you. Everything else has to be done, by hand, by you.

That means that you have to design your application so that there's some 
sort of signal that you can send it which will make it switch to the new 
version. So that takes care of your code, but what about your data? 
Well, presumably your new code might have new data layouts or other 
invariants, so you probably want to run some conversion function to port 
the running environment to your new code.

In short, _you_ have to figure out how your new code is different from 
your old code, _you_ have to write the code that converts any data 
structures or establishes any new invariants, _you_ have to check that 
this new code actually works properly, and _you_ have to design your 
application in the first place so that you can tell it to switch to the 
new code, running the conversion routine in the process.

And if you want to downgrade back to the old version? _You_ have to do 
all the work for that too. Really, the only help that Erlang is giving 
you is the ability to easily change from one chunk of code to another. 
You could probably do the self-same thing in Java, if you built your 
application so that every class has a version number, and there's some 
way to tell the application to load a set of new class files and execute 
a predetermined method on one of them. (I suspect Java's inflexibly type 
system would probably whine though...)

Erlang really gives you no assistance at all beyond the minimal level of 
"you can have two modules with the same name". And it's limited to just 
two, by the way. If you try to load a third version, anything running 
the first version is unceremoniously killed. And there's no mention of 
any way to detect whether old code is still running. (Perhaps there is, 
but I didn't see it mentioned.)

There was some talk of a packaging system, which sounds quite 
interesting, but obviously no details are described. It talks about 
being able to group modules into applications, and group applications 
into releases which can have complex dependencies, build procedures and 
installation processes. But... no details. So maybe it does 
configuration management for you, or maybe it does very little to assist 
you. I couldn't say.



The language itself has a few interesting features. Of course, being 
designed as something easily parsable by Prolog, the syntax is utterly 
horrid, it uses a kludgy preprocessor rather than actually supporting 
named constants, and so forth. It's also dynamically typed, which some 
people are presumably going to argue is somehow "necessary" because of 
what it does. I can't help thinking that a powerful type system would 
have made it /so much easier/ to make sure you got everything straight.

Just reading the document, I saw endless examples of data structures 
who's meaning is ambiguous due to the lack of types. For example, it is 
apparently impossible to tell the difference between a process that died 
because of an exception, and a process which merely sent you a message 
that happens to be a tuple containing the word "EXIT". The advice for 
this presumably being "don't do that".

It's slightly bizarre. Erlang looks for all the world like a crude 
scripting language with no safety at all and abysmal run-time 
performance. And yet, the language is designed for running network 
switches, possibly the most demanding high-performance hard-realtime 
system imaginable, and people claim it has nine 9s up-time. The only 
container types are linked lists and tuples, and yet the standard 
libraries somehow include cryptography and complex network protocols. 
Very odd...

So anyway, in spite of all the obvious bad things, there /are/ a few 
interesting bits. You get Haskell-style pattern matching, but with a 
twist. In Haskell, patterns are required to be "linear". (Don't you just 
love how that single word is used to mean a million different unrelated 
things?)

In Haskell, a variable is not allowed to appear twice in the same 
pattern. If you want to things to be equal, you must bind them to two 
different variables and then use a guard to check their equality. But 
Erlang has no such limitation. It /automatically/ desugars a non-linear 
pattern into a linear one with a guard. So there!

Erlang uses the rather baffling convention that mere variables start 
with an upper-case letter, while proper nouns such as function names or 
data points start with a lower-case letter. This makes it surprisingly 
hard to read Erlang code. (Haskell, not to mention English, obviously 
uses the opposite convention.)

I already mentioned that message receipt is by pattern matching. This 
seems like a rather powerful way to work, especially considering that a 
message that fails all patterns stays queued in the mailbox. So you can 
actually use patterns to decide what order you want to process incoming 
messages in. That seems quite sophisticated. And I already mentioned how 
you can add a time-out as well, which is obviously a very frequent 
requirement for hard-realtime systems.

In Haskell, general practise is to make all functions "total". In 
particular, all pattern matching should cover every case if possible. By 
contrast, apparently the general practise with Erlang is to /not/ cover 
cases which aren't expected to occur, and just let the thing throw an 
exception on a pattern-match failure. The rationale apparently being 
that the automatically-generated exception string is just as descriptive 
as anything that you could write by hand yourself, and it makes the code 
less cluttered.

Certainly some people have a tendency to make their functions try to 
return a valid result no matter how silly the inputs are. This is almost 
certainly not the way to write reliable software. Myself I've always 
thought that "***exception: Data.Map.lookup: key not found" is far more 
informative than "***exception: Data/Map.hs, lines 12312-12315: 
non-exhaustive patterns in case expression". In the first case, it's 
instantly obvious what the problem is, and that the bug is likely to be 
in the caller. In the second case... uh...?

Still, either way, if a key being missing is something that you're 
anticipating, you can and should handle the exception in the caller (and 
it won't matter what the text of the exception actually is). And if it's 
a high-availability system, if the exception /is/ uncaught, you have a 
program bug, and you probably want to log huge volumes of data about it, 
/and then/ restart the thing ASAP before somebody notices it went wrong!



I read the part about "behaviours" and face-palmed. The idea is that the 
standard library contains skeletons for things like "a server", and you 
supply the body code that makes it "a name server" or "a unique ID 
server" or whatever. Get this:

- The body code has to go in its own special module.

- The module has to export a specific set of functions with specific 
names and specific numbers and types of arguments returning a specific 
data structure as the result. (If only Erlang had type signatures, eh?)

- The skeleton passes an opaque "state" object between these functions, 
which the functions can update.

What does that sound like to you? Yes, congratulations, you've just 
invented object-oriented programming. But without dynamic binding or 
inheritance. (Or static typing, for that matter.) A "module which 
exports a special set of functions" is basically a *class*, and this 
"threading opaque state from function to function" is basically a 
stateful object.

Of course, modules don't have inheritance. That means that you can't use 
an abstract class to define what methods a class is supposed to have. It 
means you can't write a half-implemented class that other concrete 
classes can inherit from. But it also means that you don't have to worry 
about the nightmare of multiple inheritance either.

And there's another little twist: /Objects/ have state. But /functions/ 
passing around a mostly /immutable/ state gives the system a somewhat 
functional flavour. Indeed, in one of the example programs, a function 
receives some data, starts processing it, but if a certain condition 
happens, it /reverts to the old state/ rather than keeping the new one. 
This is jaw-droppingly hard to do if you use real objects with real 
mutable state. But in a semi-functional approach, it's quite trivial.

This whole "module with special functions" monstrosity is all the more 
weird because Erlang also has

1. first-class function names

2. lambda functions

Um, like, WTF? Why does the code have to go in its own module? Why can't 
I just pass you a tuple of function names, or even lambda functions, 
that define the callbacks? Hello??



One really neat feature of the language, which I didn't look into *too* 
deeply, is the binary operators. Given that IP datagrams and so forth 
are complicated assemblages of binary data fields with different 
alignments and so forth, packing and unpacking binary data is kind of 
crucial in networking code. Accordingly, Erlang not only allows you to 
write binary literals, but to /pattern match/ on them. Eat that!

I didn't look at it too closely, but it looks quite neat. You write an 
integer (or variable) followed by the number of bits. If you're packing, 
the requested data gets put into the appropriate places. If you're 
pattern matching, an integer obviously matches against that integer, 
while a variable causes that many bits to be copied into the variable. 
(I'm fuzzy on whether it becomes an integer or stays a binary...)

Not sure what happens if you need to write bytes backwards, as some 
broken protocols and file formats require. Is there a directive for 
that? Or do you have to do it by hand?



Many things about Erlang still remain unexplained. I still have 
absolutely no clue WTF an "atom" is. Nor have I discovered what this 
"OTP" thing that gets mentioned every 3 sentences is. I have utterly no 
idea how the "registered process names" thing works. Erlang is supposed 
to be for very high-performance systems, and yet to send any data from 
one process to another you have to /copy/ it. It seems like that can't 
possibly scale, but somehow it does. Apparently there's a distributed 
database system written in Erlang, but I didn't find any mention of 
that. And so on...

Still, I only have 1 PC, so what do I need distributed programming for? ;-)

Post a reply to this message

Am 09.03.2011 12:05, schrieb Invisible:
> OK, so Erlang is something of an interesting case. There are several
> reasons for this:
...
> - The system is supposedly insanely reliable. People toss around "nine
> 9s up-time" as if this is some sort of experimentally verified *fact*.

That uptime claim is probably not based on reliability, but on the 
ability to hot-swap code at runtime. Without such a feature, even the 
most reliable computer system (hardware + software) will have 
considerable downtime now and again just for the sake of updating the 
software.

As for reliability, with Wings 3D being written in Erlang I'd say 
apparently you can write unstable code in Erlang, too.

Post a reply to this message

>> - The system is supposedly insanely reliable. People toss around "nine
>> 9s up-time" as if this is some sort of experimentally verified *fact*.
>
> That uptime claim is probably not based on reliability, but on the
> ability to hot-swap code at runtime.

It's probably based on what the system was *designed* to do, rather than 
any *actual* measurement of its reliability.

> Without such a feature, even the
> most reliable computer system (hardware + software) will have
> considerable downtime now and again just for the sake of updating the
> software.

Well, that's true enough. But Erlang is hardly the only system to allow 
you to dynamically load code. Even C lets you do that, if you're 
masochistic enough...

> As for reliability, with Wings 3D being written in Erlang I'd say
> apparently you can write unstable code in Erlang, too.

I didn't find Wings unstable. Incomprehensible, sure. But unstable? No, 
not really.

Besides, writing unstable code is easy. It's writing *stable* code 
that's hard. ;-)

Post a reply to this message

Am 09.03.2011 12:05, schrieb Invisible:
> Not sure what happens if you need to write bytes backwards, as some
> broken protocols and file formats require. Is there a directive for
> that? Or do you have to do it by hand?

What you call "broken" is just a different convention: There is no such 
thing as a "natural" byte order, unless you're really digging as deep as 
the baseband level -  where it again breaks down to a convention issue, 
because it depends on whether multi-bit values are sent MSBit first or 
LSBit first.

Fun fact: The famous MSByte-first "network byte order" used in the 
TCP/IP protocol family does /not/ match the bit ordering of the famous 
Ethernet, which is LSBit-first. So if there is such a thing as a broken 
protocol with regard to bit/byte ordering, TCP/IP-over-Ethernet is one 
of them.

(That said, I guess a language that is deeply rooted in the world of 
network protocols will have plenty of ways to deal with byte & bit 
ordering.)

Post a reply to this message

Having worked with Erlang, here's some comments.

Invisible wrote:
> - There are many functional programming languages. (Haskell, OCaml, 
> Clean, and for some reason people keep calling Lisp "functional" too.) 
> However, Erlang is *the only* one that could be considered "commercially 
> successful", as far as I can tell.

Erlang isn't functional. It's single-assignment. The "functional" bit in 
Erlang doesn't give you benefits like it does in Haskell because there's 
still a whole bunch of non-functional operations. However, since you can 
only assign to each variable once, it means you can't have loops and you 
can't do a sequence of operations without making up new meaningless names 
for the data at each step.

If it was *actually* functional, you could put your own functions in guards, 
for example.

> - Other languages attempt to be *concurrent*, but only Erlang is 
> *distributed*.

Yep.

> - The system is supposedly insanely reliable. People toss around "nine 
> 9s up-time" as if this is some sort of experimentally verified *fact*.

It is. They've been running a commercial switching network with it for 
something like 10 years, and one of the six data centers lost power or 
something and all the machines in it for 20 minutes once, so they had a 15% 
downtime for 20 minutes out of ten years.

> - The language claims to do all sorts of wacky, far-out stuff like 
> trivial concurrency and distribution, hot-swapping running code, 
> detecting and correcting run-time errors, and so forth. I want to see 
> how it's done.

It's pretty straightforward. The rest is in libraries. The collection of 
libraries that make it easy is called OTP. The documentation on OTP is 
either non-existent or blows mooses, depending on what you can find.

> Suffice it to say, from what little I could discover, I didn't like what 
> I was seeing. Like most commercially successful languages, Erlang is 
> obtuse, complex, ugly and kludgy. Much like C, Java or anything else 
> wildly popular. It's abundantly clear that Erlang is about as 
> "functional" as Lisp (i.e., not at all). Still, people claim that 
> Ericsson's entire business is based on it, and lots of people are using 
> it, so it must have got *something* right.

Yes. It got the reliability right.

> refusing to allow processes to share state. This immediately implies 
> that if you want to send data from place to place, you must copy it all. 

No, not true. Since the data is immutable, you don't actually need to copy 
it if sender and receiver are on the same processor. It *acts* like it got 
copied, but then "X = [5, Z]" acts like it copies Z into X also.

> Then again, if one process is operating on a completely different node, 
> you are *forced* to copy data from place to place anyway. It is 
> unavoidable. So in a way, forcing you to do it all the time just makes 
> it that little bit simpler to move from local to distributed coding.

Except you're not really forced to copy stuff. And big things (like code or 
large binaries) get allocated in their own heap anyway.

> One thing that rapidly becomes clear (and doesn't seem to be mentioned 
> anywhere else) is that for Erlang, processes are about more than just 
> concurrency or distribution. They are about fault isolation.

Indeed, that was the fundamental goal. Not distributed computing, but 
reliable computing, however that may be achieved.

> Another thing that becomes painfully obvious is that writing your 
> program in Erlang does not somehow magically make it reliable. 

There are a handful of primitives that are reliable. You have to build 
everything else on top of that. For example, the spawn-link function gives 
you the basis for reliability, because there's no race condition between 
spawning the new process and linking to its error messages.

> less the same kinds of error handling that any other language provides - 
> catch/throw, etc. 

Huh. I didn't even know it provided catch/throw. I've never seen that used, 
actually.

> can make it so that related processes get auto-killed as well 
> (presumably because they can't sensibly complete without the dead 
> process), 

Yes.

 > or you can make it send a notification to some monitor
> process. The *language* itself allows notifications to be sent, but 
> nothing more. The *libraries* allow you to do sophisticated error 
> recovery, but *you* have to implement this. It doesn't happen by magic, 
> as many seem to suggest.

OTP is the libraries that make it seem like magic. However, being typical 
undocumented libraries that you learn by spending 5 years as an employee of 
Erricson, written in a language so modular you can't even figure out what 
pieces of code are part of the program and what aren't let alone read the 
code to see what it does, I've never quite figured it out beyond the very 
surface level.

> Very importantly, the system correctly handles things other than 
> software exceptions. Detecting division by zero is one thing. Detecting 
> that the machine in Australia that you were talking to just got hit by a 
> small pyroclastic flow is another. Obviously, no documentation actually 
> describes how this works.

There is documentation.

Each hardware box is running one interpreter, each of which is running one 
thread per CPU. Each interpreter keeps a TCP socket open to all the other 
interpreters it knows about, as well as sending periodic "are you there" 
messages. That TCP socket is also used to send application-level messages to 
remote processes. If the socket breaks, anything on the local system linked 
to that socket gets a notification that it broke.

There's also a separate process running on each local machine that's 
monitoring the local Erlang interpreter that will reboot the machine if the 
local Erlang interpreter stops working. (For some definition of "working" 
which isn't clear from the docs, but clearly includes answering keep-alive 
probes.)

> So how does Erlang actually work then? I mean, you can create processes. 
> OK, then what? Well, as best as I can determine, each process has a 
> "mailbox", and you can send asynchronous messages to any process that 
> you have the address of. (And these messages can contain process 
> addresses.) This works the same way both locally and remotely.

Yep.

> Sending is a non-blocking operation. Receiving is a blocking operation, 
> and it makes use of Haskell-style pattern matching (which seems like a 
> rather neat fit here). You can also time out waiting for a message - 
> something which would be really hard to implement yourself, and which is 
> probably extremely important in networking code.

Correct.

> Message sending has the curious property that message delivery is /not/ 
> guaranteed, but message ordering /is/ guaranteed. Like, WTF?

Message delivery isn't guaranteed because it's asynchronous, and the remote 
machine may crash between the time you send the message and the time the 
remote machine receives it. Ordering *is* guaranteed because if the remote 
machine crashes before it gets message #5, it won't get message #6 either.

 > I'm not
> even sure how it's possible to tell that the messages are in order 
> without being able to tell if you got all of them, but whatever. 

You can't, but the *sender* doesn't know if you got them all. The receiver does.

> very, very random to me. The thesis claims that implementing delivery 
> checks yourself is easy, but implementing ordering checks is very hard. 
> So they put the hard thing into the language, and left the easy thing up 
> to you if you need it. Um, OK?

Yep. Knowing if something is out of order is hard in a single-assignment 
language. By the time you do the pattern match, it's too late to not get the 
message out out of the queue. But the runtime can just look at the sequence 
number of messages from that other machine and pretend the message didn't 
arrive until the missing ones show up.

Given it all runs over TCP, tho, I don't think that's actually a whole lot 
of trouble.

Knowing the receiver got the message is as simple as adding code to send an 
ack each time you handle a message.

> One of the very trippy things about Erlang is that I can be sitting on 
> my PC in England, and I can just tell some Solaris server in Brazil to 
> spawn a new process. As expected, no word on how the hell this actually 
> works. I would have *thought* this means that the necessary code gets 
> beamed over the wire, but some documentation seems to suggest that you 
> have to install it yourself. And yet, you can apparently send arbitrary 
> functions as messages, so...?

The functions are merely references to installed code. The code itself has 
to already be in Brazil, and the function you send is "this module, this 
name, this version".

There's a document out there somewhere that lists the binary formatting of 
the contents of all the messages.

Starting the process remotely works because the local interpreter has (or 
establishes) a TCP link to the remote machine. All the messages for all the 
processes on your PC go over the same link to the machine in Brazil. There 
isn't one connection per communicating process.

> There's also no word on authentication. I mean, how does the machine on 
> Brazil know that I'm authorised to be executing arbitrary code on it? 

Nope, you're fucked.

> best guess is that if you're a telecoms giant, you have a private data 
> network that nobody else can physically access, and you use that to 
> control your equipment. In other words, you're authorised if you can 
> actually access the system in the first place.

Yes, exactly. Or you proxy the connection between machines through a VPN 
with authentication. Basically, you secure it at the machine-to-machine 
level, which is done outside of Erlang's source code and instead done at the 
Erlang interpreter level.

> It's a similar story when we come to code hot-swapping. Surprise, 
> surprise, it turns out that you can't just magically hot-swap code, just 
> because it's written in Erlang. No, you have to do *actual work* to make 
> this possible.

Well, sure.

> The language itself allows two (and only two) copies of the same code to 
> be loaded at once, an "old" version and a "new" version. And it provides 
> a way to switch from one to the other for a specific process. And that's 
> *all* it gives you. Everything else has to be done, by hand, by you.

The two-versions limit is because the code itself isn't GCed. They document 
that they could have GCed the code, but that would have added lots of 
overhead for something relatively rare.

If you *really* need more than two versions, first move to a "new" version 
that talks to one of N versions, then start up the N versions as appropriate.

> In short, _you_ have to figure out how your new code is different from 
> your old code, _you_ have to write the code that converts any data 
> structures or establishes any new invariants, _you_ have to check that 
> this new code actually works properly, and _you_ have to design your 
> application in the first place so that you can tell it to switch to the 
> new code, running the conversion routine in the process.

Yep.

> And if you want to downgrade back to the old version? _You_ have to do 
> all the work for that too. Really, the only help that Erlang is giving 
> you is the ability to easily change from one chunk of code to another. 
> You could probably do the self-same thing in Java, if you built your 
> application so that every class has a version number, and there's some 
> way to tell the application to load a set of new class files and execute 
> a predetermined method on one of them. (I suspect Java's inflexibly type 
> system would probably whine though...)

You'd have to be able to pass the network connections around too, remember.

> Erlang really gives you no assistance at all beyond the minimal level of 
> "you can have two modules with the same name". And it's limited to just 
> two, by the way. If you try to load a third version, anything running 
> the first version is unceremoniously killed. And there's no mention of 
> any way to detect whether old code is still running. (Perhaps there is, 
> but I didn't see it mentioned.)

There is, but it's deep in the bowels of "system" stuff.

The OTP system provides all the packaging and maintenance for the higher 
levels of management. Complaining that Erlang itself doesn't automate such 
stuff is like complaining that you need a Linux package manager because the 
file system only knows about files, not applications. :-)

> There was some talk of a packaging system, which sounds quite 
> interesting, but obviously no details are described. It talks about 
> being able to group modules into applications, and group applications 
> into releases which can have complex dependencies, build procedures and 
> installation processes. But... no details. So maybe it does 
> configuration management for you, or maybe it does very little to assist 
> you. I couldn't say.

It does a bunch of stuff, including giving you ways to roll things forward 
and back and such.

Realize that most of these "systems" you're talking about are actually 
implemented as Erlang code. So, for example, to compile code, you actually 
sit down at the REPL and invoke the compiler function. To package code, you 
sit down at the REPL and invoke the packager function. To debug code, you 
sit down at the REPL and invoke the debugging functions. Etc.

> The language itself has a few interesting features. Of course, being 
> designed as something easily parsable by Prolog, the syntax is utterly 
> horrid, it uses a kludgy preprocessor rather than actually supporting 
> named constants, and so forth. It's also dynamically typed, which some 
> people are presumably going to argue is somehow "necessary" because of 
> what it does. I can't help thinking that a powerful type system would 
> have made it /so much easier/ to make sure you got everything straight.

I agree about the syntax etc. It's quite possible a better type system would 
make it easier.

You should also check out Hermes, which is essentially the same thing with 
an astoundingly strict and high-level type system.  Indeed, Hermes is kind 
of a high-level Erlang, in that you (for example) let the compiler/runtime 
decide where to run the process and how many to create. You write a loop 
that says "read a message, process it, send an answer, loop" and it gets 
automatically parallelized. You write a process that says "read a message, 
process, send an answer to where this message came from, exit" and it gets 
turned into an in-line subroutine. Etc.

> Just reading the document, I saw endless examples of data structures 
> who's meaning is ambiguous due to the lack of types. For example, it is 
> apparently impossible to tell the difference between a process that died 
> because of an exception, and a process which merely sent you a message 
> that happens to be a tuple containing the word "EXIT". 

Well, it's possible unless you asked "turn all crashes from processes I'm 
watching into EXIT messages."

> The advice for 
> this presumably being "don't do that".

No, this is actually more like "We did that on purpose in case you want the 
supervisor to react as if you exited while you keep running." So, for 
example, you can get the supervisor to spawn a new handler process while the 
previous handler process rolls back the transaction that failed in *its* 
child process or something.

> It's slightly bizarre. Erlang looks for all the world like a crude 
> scripting language with no safety at all and abysmal run-time 
> performance. And yet, the language is designed for running network 
> switches, possibly the most demanding high-performance hard-realtime 
> system imaginable, and people claim it has nine 9s up-time. The only 
> container types are linked lists and tuples, and yet the standard 
> libraries somehow include cryptography and complex network protocols. 
> Very odd...

It's possible to link C (or anything else) into Erlang. The GUI for Erlang 
launches a separate Tcl/TK process and talks over a socket to it to do the 
drawing, for example.

In general, network switching isn't that hard real-time, because you usually 
have custom hardware to do the switching. A 5ESS (which can handle up to 
800,000 phone lines) runs on two 6800s, one of which is a hot spare. But of 
course there's a large lump of hardware that's routing the individual bytes 
here and there, so the 6800 only actually gets involved when you connect or 
disconnect, basically.

> Erlang uses the rather baffling convention that mere variables start 
> with an upper-case letter, while proper nouns such as function names or 
> data points start with a lower-case letter. This makes it surprisingly 
> hard to read Erlang code. (Haskell, not to mention English, obviously 
> uses the opposite convention.)

Erlang is the only language I've found whose syntax sucks worse than C++, 
yet it has way, way less built in than C++.

The primary problem, I think, is it started out as "let's investigate how we 
can write a million lines of code that doesn't crash."  Of course, once you 
have that million lines of code and it's running reliably, you're not going 
to go back and fix anything as trivial as the syntax.

> I already mentioned that message receipt is by pattern matching. This 
> seems like a rather powerful way to work, especially considering that a 
> message that fails all patterns stays queued in the mailbox. So you can 
> actually use patterns to decide what order you want to process incoming 
> messages in. That seems quite sophisticated. And I already mentioned how 
> you can add a time-out as well, which is obviously a very frequent 
> requirement for hard-realtime systems.

It's very handy, yes.

> In Haskell, general practise is to make all functions "total". In 
> particular, all pattern matching should cover every case if possible. By 
> contrast, apparently the general practise with Erlang is to /not/ cover 
> cases which aren't expected to occur, and just let the thing throw an 
> exception on a pattern-match failure. The rationale apparently being 
> that the automatically-generated exception string is just as descriptive 
> as anything that you could write by hand yourself, and it makes the code 
> less cluttered.

Well, that's not quite true for message pattern matching. For matching a 
pattern against a value, yes. But if you don't read messages, you leave them 
in the input buffer, which then grows and grows until you take out the 
entire interpreter. Normally you'd find the place in your code where you 
*think* you handled all the messages, and you'd put in a "if I match 
anything else, crash out" branch to the pattern match.

> - The body code has to go in its own special module.

A module is the granular unit of code update, so this makes sense.

> - The module has to export a specific set of functions with specific 
> names and specific numbers and types of arguments returning a specific 
> data structure as the result. (If only Erlang had type signatures, eh?)

There is that.

> - The skeleton passes an opaque "state" object between these functions, 
> which the functions can update.

Yep.

> What does that sound like to you? Yes, congratulations, you've just 
> invented object-oriented programming. But without dynamic binding or 
> inheritance. (Or static typing, for that matter.) A "module which 
> exports a special set of functions" is basically a *class*, and this 
> "threading opaque state from function to function" is basically a 
> stateful object.

Yep.  Except since it's actually a separate process, it's called an "actor", 
not an "object". :-)

> Of course, modules don't have inheritance. That means that you can't use 
> an abstract class to define what methods a class is supposed to have. It 
> means you can't write a half-implemented class that other concrete 
> classes can inherit from.

No. You'd instead write something that invokes functions in a module whose 
name you provided when you instantiated this module. There's actually even 
syntax for this. You don't "inherit" but you can have essentially "abstract 
modules."

> This whole "module with special functions" monstrosity is all the more 
> weird because Erlang also has
> 
> 1. first-class function names
> 
> 2. lambda functions
> 
> Um, like, WTF? Why does the code have to go in its own module? Why can't 
> I just pass you a tuple of function names, or even lambda functions, 
> that define the callbacks? Hello??

Because then you can't update the module. One of the things OTP is providing 
for you is the whole infrastructure for updating your module. That's why you 
send the "state" back to OTP and why the code has to be in a separate 
module. When OTP gets the "update" message, it invokes the new module with 
the state.

> One really neat feature of the language, which I didn't look into *too* 
> deeply, is the binary operators. 

They are pretty cool, yes. Handy, but I've seen it done better elsewhere. :-)

> Not sure what happens if you need to write bytes backwards, as some 
> broken protocols and file formats require. Is there a directive for 
> that? Or do you have to do it by hand?

I think there is. Lots of stuff can come after a second colon to control 
that sort of thing.

> Many things about Erlang still remain unexplained. I still have 
> absolutely no clue WTF an "atom" is. 

An atom is an interned string. Same as a "symbol" in LISP or Smalltalk. The 
trick is that in a message, an atom is an index into the atom table, so it's 
short regardless of how long the atom is.

> Nor have I discovered what this 
> "OTP" thing that gets mentioned every 3 sentences is. 

It's the libraries that take the basic operations Erlang supplies (like 
"load new code" or "spawn link") and turn it into things like installable 
packages.

> I have utterly no 
> idea how the "registered process names" thing works. 

A process can say "My mailbox is the registered process on server ABC for 
XYZ".  Another process can say "Give me XYZ@ABC" and get that mailbox. It 
works by shchlepping non-Erlang messages between the erlang processes, just 
like the keep-alives do.

 > Erlang is supposed
> to be for very high-performance systems, 

Not really.

> Apparently there's a distributed 
> database system written in Erlang, but I didn't find any mention of 

Mnesia. They were originally going to call it Amnesia, but someone pointed 
out that naming a database system after an illness characterized by 
forgetting things is probably a bad idea.

It's a cute little system, but it has its flaws due to being implemented on 
top of a single-assignment language and keeping everything in memory. For 
example, if the node crashes and you have a gigabyte of data in the table of 
that process, it has to go through reading all the logged updates and 
applying them to the tree in memory before it'll actually start answering 
queries. I guess if you can afford several machines running the same 
database entirely out of RAM, it's not much of a problem.

It's basically a distributed transactional system on top of the built-in 
database tables whose TLA name escapes me at the moment.

http://www.erlang.org/doc/man/mnesia.html

Note that all these things, OTP and Mnesia, along with stuff like the 
compiler, the debugger, the profiler, the package manager, etc etc etc, are 
all documented as "here's the functions you use to do it in Erlang." Almost 
none of these things are actually command-line tools.

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

clipka wrote:
> That uptime claim is probably not based on reliability, but on the 
> ability to hot-swap code at runtime.

It's both.

> As for reliability, with Wings 3D being written in Erlang I'd say 
> apparently you can write unstable code in Erlang, too.

Yes. But Erlang gives you the tools to recover from the instability without 
taking down the whole system.

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

Invisible wrote:
> Well, that's true enough. But Erlang is hardly the only system to allow 
> you to dynamically load code. Even C lets you do that, if you're 
> masochistic enough...

No, technically, it isn't C doing the dyanmic loading. :-)

In any case, the benefit that Erlang gives you in terms of dynamically 
loading the code is (A) it's automatic, and (B) it lets the old code finish 
up while the new code is running. It's like letting currently-running web 
pages finish on the server while you're serving the new web pages to someone 
else. Yeah, we pretty much can solve that now, but it was a necessary 
component to the whole reliability enchalada.

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

On 09/03/2011 06:22 PM, Darren New wrote:
> Having worked with Erlang, here's some comments.

In what capacity are you "working with Erlang"? As I understand it, the 
main selling points of the language are reliability and distribution. 
While I doubt anyone actually *wants* unreliable software, there can't 
be too many problems that require distributed programming.

> Erlang isn't functional. It's single-assignment. The "functional" bit in
> Erlang doesn't give you benefits like it does in Haskell because there's
> still a whole bunch of non-functional operations. However, since you can
> only assign to each variable once, it means you can't have loops and you
> can't do a sequence of operations without making up new meaningless
> names for the data at each step.

Erlang is partially functional, in that functions are apparently 
first-class, and it's got single-assignment and immutable data 
structures. On the other hand, it doesn't enforce referential 
transparency; you can put side-effects anywhere you like. The document I 
read claims that it's considered beneficial to limit side-effects to 
just a few places in the codebase, and write most of the code in a 
"pure" style - which is exactly the same as for Haskell. It's just that 
the language itself doesn't /enforce/ any kind of strong separation 
between pure and impure code.

> If it was *actually* functional, you could put your own functions in
> guards, for example.

Wait - you *can't* do this already?? o_O

Why on earth would you design it this way?

>> - The system is supposedly insanely reliable. People toss around "nine
>> 9s up-time" as if this is some sort of experimentally verified *fact*.
>
> It is. They've been running a commercial switching network with it for
> something like 10 years, and one of the six data centers lost power or
> something and all the machines in it for 20 minutes once, so they had a
> 15% downtime for 20 minutes out of ten years.

One data point is scientific fact?

> It's pretty straightforward. The rest is in libraries.

Yeah, as I quickly found out.

> The collection of
> libraries that make it easy is called OTP. The documentation on OTP is
> either non-existent or blows mooses, depending on what you can find.

Just like Haskell, then. :-}

>> refusing to allow processes to share state. This immediately implies
>> that if you want to send data from place to place, you must copy it all.
>
> No, not true. Since the data is immutable, you don't actually need to
> copy it if sender and receiver are on the same processor.

OK, fair enough.

(Presumably both processes have to be running within the same logical 
"node", but then again also presumably you wouldn't have more than one 
node on the same box except for test purposes anyway...)

>> One thing that rapidly becomes clear (and doesn't seem to be mentioned
>> anywhere else) is that for Erlang, processes are about more than just
>> concurrency or distribution. They are about fault isolation.
>
> Indeed, that was the fundamental goal. Not distributed computing, but
> reliable computing, however that may be achieved.

As I say, this doesn't really seem to be spelled out anywhere else that 
I could find, and it seems a rather crucial detail.

>> less the same kinds of error handling that any other language provides
>> - catch/throw, etc.
>
> Huh. I didn't even know it provided catch/throw. I've never seen that
> used, actually.

Yeah they do. It's in there. Apparently the catch functions turn an 
exception into a tuple begining with 'EXIT' and followed by some stuff. 
Using throw you can make catch return an arbitrary data structure. 
(Quite how you tell the difference between an exception and a normal 
result is left to the imagination... It looks like you can use throw to 
make it look like the catch block exited normally.)

> OTP is the libraries that make it seem like magic. However, being
> typical undocumented libraries that you learn by spending 5 years as an
> employee of Erricson, written in a language so modular you can't even
> figure out what pieces of code are part of the program and what aren't
> let alone read the code to see what it does, I've never quite figured it
> out beyond the very surface level.

I'm glad it's not just me...

> There is documentation.
>
> Each hardware box is running one interpreter, each of which is running
> one thread per CPU. Each interpreter keeps a TCP socket open to all the
> other interpreters it knows about, as well as sending periodic "are you
> there" messages. That TCP socket is also used to send application-level
> messages to remote processes. If the socket breaks, anything on the
> local system linked to that socket gets a notification that it broke.
>
> There's also a separate process running on each local machine that's
> monitoring the local Erlang interpreter that will reboot the machine if
> the local Erlang interpreter stops working. (For some definition of
> "working" which isn't clear from the docs, but clearly includes
> answering keep-alive probes.)

Right. So it just sends heartbeat messages? (I presume the bit about 
rebooting the machine is optional; you wouldn't want your PC to reboot 
just because Wings 3D crashed, for example.)

>> Message sending has the curious property that message delivery is
>> /not/ guaranteed, but message ordering /is/ guaranteed. Like, WTF?
>
> Message delivery isn't guaranteed because it's asynchronous, and the
> remote machine may crash between the time you send the message and the
> time the remote machine receives it. Ordering *is* guaranteed because if
> the remote machine crashes before it gets message #5, it won't get
> message #6 either.

You mean that if message #5 doesn't arrive, message #6 is actually 
guaranteed not to arrive?

>> very, very random to me. The thesis claims that implementing delivery
>> checks yourself is easy, but implementing ordering checks is very
>> hard. So they put the hard thing into the language, and left the easy
>> thing up to you if you need it. Um, OK?
>
> Yep. Knowing if something is out of order is hard in a single-assignment
> language.

OK...

> Given it all runs over TCP, tho, I don't think that's actually a whole
> lot of trouble.

TCP guarantees both delivery and ordering. Unless the connection 
actually breaks. So is that all Erlang is saying? That it if one of the 
machines catches fire, Erlang can't guarantee that messages will be 
delivered to it?

These people are /serious/ about reliability. o_O

> Knowing the receiver got the message is as simple as adding code to send
> an ack each time you handle a message.

Network protocols 101: It's not that simple. It is possible for the 
message to arrive, but the acknowledgement itself to fail to be delivered.

Now, if you're running on TCP, so that ordering and delivery are 
guaranteed (unless there's no transport or no endpoint), the only way 
this can happen is if the transport or the endpoint dies after the 
message is received, which is probably just as bad as the message not 
being delivered...

>> One of the very trippy things about Erlang is that I can be sitting on
>> my PC in England, and I can just tell some Solaris server in Brazil to
>> spawn a new process. As expected, no word on how the hell this
>> actually works. I would have *thought* this means that the necessary
>> code gets beamed over the wire, but some documentation seems to
>> suggest that you have to install it yourself. And yet, you can
>> apparently send arbitrary functions as messages, so...?
>
> The functions are merely references to installed code. The code itself
> has to already be in Brazil, and the function you send is "this module,
> this name, this version".

I did think for a moment that you could send a function this way. Once 
you've compiled your module, every lambda function it contains must 
already exist in the compiled code somewhere. High-order functions make 
the program more complicated, of course. You can't necessarily point to 
just *one* function. It might be that you call several functions to 
construct the final function object. But even that is likely to be more 
compact than the actual executable code.

Thing is, I can open the REPL and type in a function, and then ask to 
spawn that on a remote node. How does *that* work?!

> There's a document out there somewhere that lists the binary formatting
> of the contents of all the messages.

I'll bet. No doubt somebody has written libraries for other languages to 
speak the same wire protocol.

> Starting the process remotely works because the local interpreter has
> (or establishes) a TCP link to the remote machine. All the messages for
> all the processes on your PC go over the same link to the machine in
> Brazil. There isn't one connection per communicating process.

Given that Erlang supports creating billions of processes and there's 
only a few thousand possible TCP ports... yeah, I figured that's how it 
would work.

I notice that connecting one Erlang node to another causes the two nodes 
to connect to *all* the nodes that each endpoint knows about. I'm not 
actually sure why though. You would think that a fully interconnected 
mesh would quickly exhaust the supply of TCP ports too.

>> There's also no word on authentication. I mean, how does the machine
>> on Brazil know that I'm authorised to be executing arbitrary code on it?
>
> Nope, you're fucked.
>
>> In other words, you're authorised if you can
>> actually access the system in the first place.
>
> Yes, exactly.

Ouch.

>> It's a similar story when we come to code hot-swapping. Surprise,
>> surprise, it turns out that you can't just magically hot-swap code,
>> just because it's written in Erlang. No, you have to do *actual work*
>> to make this possible.
>
> Well, sure.

You say "sure", but the documentation sings about how reliable Erlang is 
and makes it sound like all this stuff just happens by magic, because 
Erlang is so wonderful. The truth is actually that Erlang just 
implements the minimal features required to let _you_ build complex, 
robust, distributed applications.

> The two-versions limit is because the code itself isn't GCed. They
> document that they could have GCed the code, but that would have added
> lots of overhead for something relatively rare.

Fair enough.

>> Really, the only help that Erlang is giving
>> you is the ability to easily change from one chunk of code to another.
>> You could probably do the self-same thing in Java, if you built your
>> application so that [...]
>
> You'd have to be able to pass the network connections around too, remember.

You'd have to be able to do a lot of things. It would require careful 
application design, but it doesn't look especially difficult. (To get 
the framework done, anyway. Making the application itself sounds really 
hard, in Java or Erlang.)

>> Erlang really gives you no assistance at all beyond the minimal level
>> of "you can have two modules with the same name". And it's limited to
>> just two, by the way. If you try to load a third version, anything
>> running the first version is unceremoniously killed. And there's no
>> mention of any way to detect whether old code is still running.
>> (Perhaps there is, but I didn't see it mentioned.)
>
> There is, but it's deep in the bowels of "system" stuff.
>
> The OTP system provides all the packaging and maintenance for the higher
> levels of management. Complaining that Erlang itself doesn't automate
> such stuff is like complaining that you need a Linux package manager
> because the file system only knows about files, not applications. :-)

I don't mind which level the assistance is implemented at, just so long 
as there *is* some. ;-)

>> There was some talk of a packaging system, which sounds quite
>> interesting, but obviously no details are described.
>
> It does a bunch of stuff, including giving you ways to roll things
> forward and back and such.

Yeah, that's what I figured. Handling this stuff manually for more than 
a few dozen nodes sounds like the exact opposite of "reliable".

> Realize that most of these "systems" you're talking about are actually
> implemented as Erlang code.

Hey, it worked for Smalltalk. ;-)

> So, for example, to compile code, you
> actually sit down at the REPL and invoke the compiler function.

I wish the Haskell compiler could do this...

> I agree about the syntax etc. It's quite possible a better type system
> would make it easier.

Escaped experiment, eh? ;-)

> You should also check out Hermes, which is essentially the same thing
> with an astoundingly strict and high-level type system.

Heh, there is never enough free time. ;-)

>> Just reading the document, I saw endless examples of data structures
>> who's meaning is ambiguous due to the lack of types. For example, it
>> is apparently impossible to tell the difference between a process that
>> died because of an exception, and a process which merely sent you a
>> message that happens to be a tuple containing the word "EXIT".
>
> Well, it's possible unless you asked "turn all crashes from processes
> I'm watching into EXIT messages."

The advice presumably being that workers should only receive "real" 
messages, and supervisors should only receive exceptions. Then there can 
be no ambiguity. (Until the supervisor's supervisor asks it to do 
something...)

Actually, that's a point. Each process has *one* mailbox. Not one 
mailbox for each process it's communicating with. Just *one*. So you 
have to manually figure out who each message came from - usually by 
having the sender put a reply address in manually.

>> The advice for this presumably being "don't do that".
>
> No, this is actually more like "We did that on purpose in case you want
> the supervisor to react as if you exited while you keep running."

To quote the document, "it's a feature, not a bug!"

>> It's slightly bizarre. Erlang looks for all the world like a crude
>> scripting language with no safety at all and abysmal run-time
>> performance. And yet, the language is designed for running network
>> switches, possibly the most demanding high-performance hard-realtime
>> system imaginable, and people claim it has nine 9s up-time. The only
>> container types are linked lists and tuples, and yet the standard
>> libraries somehow include cryptography and complex network protocols.
>> Very odd...
>
> It's possible to link C (or anything else) into Erlang. The GUI for
> Erlang launches a separate Tcl/TK process and talks over a socket to it
> to do the drawing, for example.

So you're saying that all the "interesting" stuff like the SSL 
implementation is actually just an off-the-shelf C library?

I never did understand what's so great about Tk. It looks horrid, it's 
really hard to use, and it's almost impossible to get the layout you want...

> In general, network switching isn't that hard real-time, because you
> usually have custom hardware to do the switching. A 5ESS (which can
> handle up to 800,000 phone lines) runs on two 6800s, one of which is a
> hot spare. But of course there's a large lump of hardware that's routing
> the individual bytes here and there, so the 6800 only actually gets
> involved when you connect or disconnect, basically.

Wait, a 6800? As in, the chip *before* the obsolete 68000?

Damn. If the actual switching is in hardware, that the heck do you need 
the CPU for at all?

> Erlang is the only language I've found whose syntax sucks worse than
> C++, yet it has way, way less built in than C++.

Erlang: 0wned.

> The primary problem, I think, is it started out as "let's investigate
> how we can write a million lines of code that doesn't crash." Of course,
> once you have that million lines of code and it's running reliably,
> you're not going to go back and fix anything as trivial as the syntax.

Yeah. It's an escaped experiment. And now it's too late. TOO LATE!!! >_<

Actually, one of the reasons I wanted to learn about Erlang was to see 
if I can design something that does the same thing, but isn't insane.

>> By contrast, apparently the general practise with Erlang is to /not/
>> cover cases which aren't expected to occur, and just let the thing
>> throw an exception on a pattern-match failure.
>
> Well, that's not quite true for message pattern matching. For matching a
> pattern against a value, yes. But if you don't read messages, you leave
> them in the input buffer, which then grows and grows until you take out
> the entire interpreter. Normally you'd find the place in your code where
> you *think* you handled all the messages, and you'd put in a "if I match
> anything else, crash out" branch to the pattern match.

Well, yeah, I meant for pattern matching against a value. Receiving 
messages as a bit special.

>> What does that sound like to you? Yes, congratulations, you've just
>> invented object-oriented programming.
>
> Yep. Except since it's actually a separate process, it's called an
> "actor", not an "object". :-)

That's possibly the most irritating thing about the gen_server example. 
It explains what each line of code is doing, but neglects to mention the 
fact that one half of the code is potentially running on another 
continent to the other - which, let's face is, is the entire point of 
making *remote* procedure calls. :-P

>> Of course, modules don't have inheritance.
>
> No. You'd instead write something that invokes functions in a module
> whose name you provided when you instantiated this module. There's
> actually even syntax for this.

Yeah. It's just a tiny bit more work, that's all.

>> This whole "module with special functions" monstrosity is all the more
>> weird because Erlang also has
>>
>> 1. first-class function names
>>
>> 2. lambda functions
>>
>> Um, like, WTF? Why does the code have to go in its own module? Why
>> can't I just pass you a tuple of function names, or even lambda
>> functions, that define the callbacks? Hello??
>
> Because then you can't update the module. One of the things OTP is
> providing for you is the whole infrastructure for updating your module.
> That's why you send the "state" back to OTP and why the code has to be
> in a separate module. When OTP gets the "update" message, it invokes the
> new module with the state.

OK. So that's why the state is explicitly threaded (quite apart from 
Erlang apparently having no other way to handle running state). That 
still doesn't explain why I can't put several related behaviours into a 
single module. Perhaps I *want* to replace them all at the same time.

>> Many things about Erlang still remain unexplained. I still have
>> absolutely no clue WTF an "atom" is.
>
> An atom is an interned string. Same as a "symbol" in LISP or Smalltalk.
> The trick is that in a message, an atom is an index into the atom table,
> so it's short regardless of how long the atom is.

I never really understood what an interned string is. (And hence what 
the difference between a Smalltalk string and a symbol is.)

So you're saying that the compiler replaces all occurrences of an atom 
with a pointer to a single instance of the string? (And hence, you can 
compare atoms for equity by simple pointer equity rather than string 
comparison.)

>> Nor have I discovered what this "OTP" thing that gets mentioned every
>> 3 sentences is.
>
> It's the libraries that take the basic operations Erlang supplies (like
> "load new code" or "spawn link") and turn it into things like
> installable packages.

Right, OK. So what does it mean when they say "the compiler is an OTP 
application"?

>> I have utterly no idea how the "registered process names" thing works.
>
> A process can say "My mailbox is the registered process on server ABC
> for XYZ". Another process can say "Give me XYZ@ABC" and get that
> mailbox. It works by shchlepping non-Erlang messages between the erlang
> processes, just like the keep-alives do.

OK, so, I can register a PID with a particular name. Is that name local 
to just that node? Does every node in the system know about it? How do 
you avoid registering a name that some unrelated application has already 
registered? If I ask for a particular name, how does it figure out which 
node that's on? If a node goes down, does the name get taken out of 
service? How do you handle the same service being available from several 
nodes? (For load-balancing or fault tolerance, perhaps.)

>> Erlang is supposed to be for very high-performance systems,
>
> Not really.

Hmm. It certainly seems to be.

>> Apparently there's a distributed database system written in Erlang,
>> but I didn't find any mention of
>
> Mnesia. They were originally going to call it Amnesia, but someone
> pointed out that naming a database system after an illness characterized
> by forgetting things is probably a bad idea.

The name still looks wonky to me. ;-)

> It's a cute little system, but it has its flaws due to being implemented
> on top of a single-assignment language and keeping everything in memory.

Hmm. Sounds... fun.

> It's basically a distributed transactional system on top of the built-in
> database tables whose TLA name escapes me at the moment.

Right.

> Note that all these things, OTP and Mnesia, along with stuff like the
> compiler, the debugger, the profiler, the package manager, etc etc etc,
> are all documented as "here's the functions you use to do it in Erlang."
> Almost none of these things are actually command-line tools.

Nothing wrong with that. The question is how much detail the 
documentation explains. ;-)

Post a reply to this message

>> Apparently there's a distributed database system written in Erlang,
>> but I didn't find any mention of
>
> Mnesia. They were originally going to call it Amnesia, but someone
> pointed out that naming a database system after an illness characterized
> by forgetting things is probably a bad idea.
>
> It's a cute little system, but it has its flaws.

> http://www.erlang.org/doc/man/mnesia.html

An object/relational hybrid "suitable for telecommunications applications"?

It says it has "extremely fast real-time searches", but it also says 
that the query language is "an add-on library"?

Reading the rest of the documentation just made my head hurt. I 
particularly love how a "table" can be "set", "ordered set" or "bag", 
but only the first and last option are explained.

A reference manual is no way to learn how to use a complex system.

Post a reply to this message

Invisible wrote:
> On 09/03/2011 06:22 PM, Darren New wrote:
>> Having worked with Erlang, here's some comments.
> 
> In what capacity are you "working with Erlang"? 

I'm not, currently.

> While I doubt anyone actually *wants* unreliable software, there can't 
> be too many problems that require distributed programming.

Unless you want to listen to radio stations in several different cities and 
integrate the results. Altho I wound up writing my own stuff for that and 
not using Erlang, that was the experiment.

> Erlang is partially functional, in that functions are apparently 
> first-class, and it's got single-assignment and immutable data 
> structures. 

Functions in C are first class. That doesn't mean it's functional. The 
immutable data structures come from the single-assignment, except where the 
data structures aren't immutable, of which there are several.

> On the other hand, it doesn't enforce referential 
> transparency; you can put side-effects anywhere you like.

Well, that's the killer. To me, if every statement can have side-effects, 
then it's not a functional language. It's just single-assignment. You have 
none of the benefits and all of the draw-backs of a functional language in 
Erlang.

 > The document I
> read claims that it's considered beneficial to limit side-effects to 
> just a few places in the codebase, 

Except that message sending and message receiving are both side-effects as 
is spawning or linking another process, so that works pretty poorly. Plus, 
as I said, having the side-effects means you cannot (for example) use your 
own functions in guards. Essentially, nothing is actually functional in the 
language *except* assignment and a handful of built-ins. (Not even all the 
built-ins, for that matter.)

> the language itself doesn't /enforce/ any kind of strong separation 
> between pure and impure code.

For some reason, the Erlang enthusiasts think the crappy syntax and the 
single-assignment procedures are intentional and beneficial properties of 
the language. Like any niche language, the fanboi's refuse to accept that 
any compromise was made during its design.

>> If it was *actually* functional, you could put your own functions in
>> guards, for example.
> 
> Wait - you *can't* do this already?? o_O
> Why on earth would you design it this way?

Because your own functions aren't referentially transparent, because the 
language isn't actually functional, even at that level. The functions in a 
guard are the ones that *are* functional. Not even all the built-in 
functions can go in guards.

Actually, in the documentation, I think there's a note on each built in 
function that says whether you can use it in a guard or not.

>>> - The system is supposedly insanely reliable. People toss around "nine
>>> 9s up-time" as if this is some sort of experimentally verified *fact*.
>>
>> It is. They've been running a commercial switching network with it for
>> something like 10 years, and one of the six data centers lost power or
>> something and all the machines in it for 20 minutes once, so they had a
>> 15% downtime for 20 minutes out of ten years.
> 
> One data point is scientific fact?

It's a scientific fact that you can achieve nine 9's of uptime over the 
course of a decade with programs written in Erlang, yes.  Pretty sure that's 
really, really hard to do with any other programming language.

> (Presumably both processes have to be running within the same logical 
> "node", but then again also presumably you wouldn't have more than one 
> node on the same box except for test purposes anyway...)

You start up the run-time with a command-line argument that says "Hey, I'm a 
node in a distributed program". Wings3D doesn't do this, so it's not a node.

> As I say, this doesn't really seem to be spelled out anywhere else that 
> I could find, and it seems a rather crucial detail.

It's kind of in all the books and stuff. I think the PhD thesis is presumed 
required reading, if nothing else.

>>> less the same kinds of error handling that any other language provides
>>> - catch/throw, etc.
>>
>> Huh. I didn't even know it provided catch/throw. I've never seen that
>> used, actually.
> 
> Yeah they do. It's in there. Apparently the catch functions turn an 
> exception into a tuple begining with 'EXIT' and followed by some stuff. 
> Using throw you can make catch return an arbitrary data structure. 

Ah, I see. Yes, I had forgotten about that.

> (Quite how you tell the difference between an exception and a normal 
> result is left to the imagination... 

Well, an exception will start with EXIT, while a regular result is unlikely 
to, unless you specifically confuse things that way.

> It looks like you can use throw to 
> make it look like the catch block exited normally.)

Yes. Basically, it lets you return early.

> Right. So it just sends heartbeat messages? (I presume the bit about 
> rebooting the machine is optional; you wouldn't want your PC to reboot 
> just because Wings 3D crashed, for example.)

Yes, it's a separate monitor process. It sends heartbeats as well as watches 
for a couple other simple things.

>>> Message sending has the curious property that message delivery is
>>> /not/ guaranteed, but message ordering /is/ guaranteed. Like, WTF?
>>
>> Message delivery isn't guaranteed because it's asynchronous, and the
>> remote machine may crash between the time you send the message and the
>> time the remote machine receives it. Ordering *is* guaranteed because if
>> the remote machine crashes before it gets message #5, it won't get
>> message #6 either.
> 
> You mean that if message #5 doesn't arrive, message #6 is actually 
> guaranteed not to arrive?

If message #5 didn't arrive because the receiving machine crashed, yes, 
message #6 isn't going to be received either. If message #5 didn't arrive 
because the network switch died, then #6 isn't going to be delivered either.

> TCP guarantees both delivery and ordering. Unless the connection 
> actually breaks. So is that all Erlang is saying? That it if one of the 
> machines catches fire, Erlang can't guarantee that messages will be 
> delivered to it?

Yes, exactly.  More precisely, you can't tell if the message was received 
and processed or not.

> These people are /serious/ about reliability. o_O

Well, yes, that's how you get 5 minutes of downtime over the course of 10 years.

>> Knowing the receiver got the message is as simple as adding code to send
>> an ack each time you handle a message.
> 
> Network protocols 101: It's not that simple. It is possible for the 
> message to arrive, but the acknowledgement itself to fail to be delivered.

If the ack fails to be delivered, it's because the receiving machine (nee 
sender) has crashed and hence doesn't need to see the ack.

Well, not quite. If the network between the machines dies, each machine is 
going to think the other crashed. This makes a mess. Mnesia has all kinds of 
crap in place to recover from this, especially given that you might make 
independent updates to the same table in two places.

> Now, if you're running on TCP, so that ordering and delivery are 
> guaranteed (unless there's no transport or no endpoint), the only way 
> this can happen is if the transport or the endpoint dies after the 
> message is received, which is probably just as bad as the message not 
> being delivered...

Right. The point of their warning is, you have to deal with messages not 
making it one direction or the other. You don't have to deal with the 
messages being out of order. You have to deal with the fact that if you tell 
a remote machine to do something, it might not happen. You don't have to 
deal with the fact that if you tell a remote machine to do two things, it 
might do them in a different order.

> Thing is, I can open the REPL and type in a function, and then ask to 
> spawn that on a remote node. How does *that* work?!

Hmmm. Actually, I think perhaps it *is* willing to ship functions around. 
The details escape me at the moment. Maybe you can send functions but not 
modules? I don't honestly remember.

> I'll bet. No doubt somebody has written libraries for other languages to 
> speak the same wire protocol.

I don't know. Probably, but I'm not sure why you would.

> I notice that connecting one Erlang node to another causes the two nodes 
> to connect to *all* the nodes that each endpoint knows about. 

Yes, unless you tell it not to.

> I'm not actually sure why though. 

Reliability. ;-)

> You would think that a fully interconnected 
> mesh would quickly exhaust the supply of TCP ports too.

Unless you have more than 50,000 machines, unlikely.

>>> In other words, you're authorised if you can
>>> actually access the system in the first place.
>>
>> Yes, exactly.
> 
> Ouch.

One of the OTP functions shows how to authenticate and encrypt the 
connection en Erlang code, IIRC. That's also one of the tutorials.

Imagine a room full of a few thousand processors. One outgoing internet 
connection running thru a firewall that's happy to VPN anything going out to 
a known location, and which drops anything going in or out from an unknown 
IP.  Why encrypt everything?

Actually, I had forgotten. There is a "cookie" you can set, and anyone 
connecting needs the same "cookie". So there is a minimal security level, 
not unlike the similar feature in X servers.

> You say "sure", but the documentation sings about how reliable Erlang is 
> and makes it sound like all this stuff just happens by magic, because 
> Erlang is so wonderful. The truth is actually that Erlang just 
> implements the minimal features required to let _you_ build complex, 
> robust, distributed applications.

What, you're believing fanboi's?

If you want the version that actually *does* magically distribute and make 
reliable your code, you want to read about Hermes, a language designed by Strom.

>> The two-versions limit is because the code itself isn't GCed. They
>> document that they could have GCed the code, but that would have added
>> lots of overhead for something relatively rare.
> 
> Fair enough.

Plus, you should be dealing with the occasional "process suddenly ends for 
no reason" exception robustly anyway. :-)

> You'd have to be able to do a lot of things. It would require careful 
> application design, but it doesn't look especially difficult. (To get 
> the framework done, anyway. Making the application itself sounds really 
> hard, in Java or Erlang.)

Heck. Honestly, you can do it in Ada.  Ada has facilities for declaring some 
library code as capable of being hot loaded and unloaded.

> I don't mind which level the assistance is implemented at, just so long 
> as there *is* some. ;-)

I found it annoying that there's no global roadmap. So you read about 
something and it says "If X happens, a message is sent to the error logger." 
  WTF is the error logger? Where's the documentation. Then you track that 
down, and it's all talking about bits and pieces that are also not 
documented, etc, with the assumption that you have a bunch of people who 
wrote the error logger hanging around to tell you how to find where the 
errors got logged and such.

>> Realize that most of these "systems" you're talking about are actually
>> implemented as Erlang code.
> 
> Hey, it worked for Smalltalk. ;-)

Yeah, basically the same idea, except instead of an "image" you actually 
have a bunch of running processes.

>> I agree about the syntax etc. It's quite possible a better type system
>> would make it easier.
> 
> Escaped experiment, eh? ;-)

That's exactly what it was. Except that by the time it escaped, there were 
millions of lines of production code written in it, so you couldn't even 
reasonably fix it.

>> You should also check out Hermes, which is essentially the same thing
>> with an astoundingly strict and high-level type system.
> 
> Heh, there is never enough free time. ;-)

I don't think the book is available any more, either. I should scan mine.

> The advice presumably being that workers should only receive "real" 
> messages, and supervisors should only receive exceptions. Then there can 
> be no ambiguity. (Until the supervisor's supervisor asks it to do 
> something...)

Well, more like, normal worker processes shouldn't communicate with messages 
that start with the atom "EXIT". It's not that hard. It's like saying "don't 
make closing the TCP connection the confirmation."  There's only one or two 
patterns that the error messages use. You shouldn't use "Close" as the name 
of the routine to invoke when you want your object to keep running, either.

> Actually, that's a point. Each process has *one* mailbox. Not one 
> mailbox for each process it's communicating with. Just *one*. So you 
> have to manually figure out who each message came from - usually by 
> having the sender put a reply address in manually.

Yep.

Hermes gives you as many mailboxes as you want, with as many connections to 
each mailbox as you want, and you can send both connections and mailboxes in 
messages to other processes. So if you want hot-swap code, write your 
processes as being capable of packaging up its mailboxes and sending them to 
a new process.

> So you're saying that all the "interesting" stuff like the SSL 
> implementation is actually just an off-the-shelf C library?

It's possible.

> I never did understand what's so great about Tk. It looks horrid, it's 
> really hard to use, and it's almost impossible to get the layout you 
> want...

Well, no, it's really easy to use, it's easy to get a good-looking layout, 
and it only looks horrid because it looked cutting-edge 10 years ago and 
nobody ever improved the look of it since then. (Altho they've worked on 
that more lately.)

That's exactly why half a dozen non-Tcl languages actually went so far as to 
embed a Tcl interpreter into their code in order to be able to use Tk. I'm 
pretty sure it's not Tk that's at fault here. ;-)

>> In general, network switching isn't that hard real-time, because you
>> usually have custom hardware to do the switching. A 5ESS (which can
>> handle up to 800,000 phone lines) runs on two 6800s, one of which is a
>> hot spare. But of course there's a large lump of hardware that's routing
>> the individual bytes here and there, so the 6800 only actually gets
>> involved when you connect or disconnect, basically.
> 
> Wait, a 6800? As in, the chip *before* the obsolete 68000?

Yes.

> Damn. If the actual switching is in hardware, that the heck do you need 
> the CPU for at all?

For deciding what to connect. You pick up the phone? The 6800 connects your 
phone to the dial-tone generator. It's not sitting there generating the 
waveform in software. You push a button? The 6800 routes your line to the 
DTMF decoder, which signals the 6800 each time you let go of the tone. When 
you've dialed enough numbers, the 6800 figures out what line you're supposed 
to connect to and plugs you through.

> Actually, one of the reasons I wanted to learn about Erlang was to see 
> if I can design something that does the same thing, but isn't insane.

Check out Hermes first. :-)

> Well, yeah, I meant for pattern matching against a value. Receiving 
> messages as a bit special.

Right.

> OK. So that's why the state is explicitly threaded (quite apart from 
> Erlang apparently having no other way to handle running state). That 
> still doesn't explain why I can't put several related behaviours into a 
> single module. Perhaps I *want* to replace them all at the same time.

I don't know. Go work for Erricson. ;-)

>>> Many things about Erlang still remain unexplained. I still have
>>> absolutely no clue WTF an "atom" is.
>>
>> An atom is an interned string. Same as a "symbol" in LISP or Smalltalk.
>> The trick is that in a message, an atom is an index into the atom table,
>> so it's short regardless of how long the atom is.
> 
> I never really understood what an interned string is. (And hence what 
> the difference between a Smalltalk string and a symbol is.)

I explained this for Warp a couple weeks ago. An interned value is a 
reference value where reference equality equals value equality.

Or, in normal speak, an interned string is one where there's only one copy 
of any given string, so you can compare the addresses where two strings are 
stored and tell if they're equal.

In Erlang, you can think of an atom as one element of a giant global enum of 
all the atoms you've ever used in your program. It's nothing more than an 
index into a table that happens to be readable.

> So you're saying that the compiler replaces all occurrences of an atom 
> with a pointer to a single instance of the string? (And hence, you can 
> compare atoms for equity by simple pointer equity rather than string 
> comparison.)

Basically, that's what an interned string is. An "atom" is implemented as an 
interned string, but it's syntactically different. It's ... an atom. It's a 
string treated as a single atomic value. You can't ask for the third 
character of an atom, for example.

>>> Nor have I discovered what this "OTP" thing that gets mentioned every
>>> 3 sentences is.
>>
>> It's the libraries that take the basic operations Erlang supplies (like
>> "load new code" or "spawn link") and turn it into things like
>> installable packages.
> 
> Right, OK. So what does it mean when they say "the compiler is an OTP 
> application"?

It means the compiler is a function in the OTP libraries that you invoke in 
order to compile the code.

> OK, so, I can register a PID with a particular name. Is that name local 
> to just that node? Does every node in the system know about it? How do 
> you avoid registering a name that some unrelated application has already 
> registered? If I ask for a particular name, how does it figure out which 
> node that's on? If a node goes down, does the name get taken out of 
> service? How do you handle the same service being available from several 
> nodes? (For load-balancing or fault tolerance, perhaps.)

I don't remember all this stuff. Registering as a known name passes your 
mailbox and the name to a process that proceeds to distribute it and answer 
queries about it. You can look at the documentation for it to see what it does.

>> It's basically a distributed transactional system on top of the built-in
>> database tables whose TLA name escapes me at the moment.
> 
> Right.

ETS is the name of the sub-system functions.

>> Note that all these things, OTP and Mnesia, along with stuff like the
>> compiler, the debugger, the profiler, the package manager, etc etc etc,
>> are all documented as "here's the functions you use to do it in Erlang."
>> Almost none of these things are actually command-line tools.
> 
> Nothing wrong with that. The question is how much detail the 
> documentation explains. ;-)

The other problem is that all the documentation assumes you've read the 
previous documentation and knows what they're talking about, but it doesn't 
actually give you pointers to any of the previous documentation.

So if you want to use mnesia, you read mnesia, but it assumes you know what 
ets and gen_server are and etc. So mnesia will have a function and say "this 
only works on ram_raw tables" or "that function starts a new server" and 
assume that you know "ram_raw" is something that ets implements (not Mnesia) 
and that a 'server' in this context specifically means a gen_server module.

You'd like mnesia, tho. It's a lot like STM in some ways.

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

Invisible wrote:
> An object/relational hybrid "suitable for telecommunications applications"?

By which they mean reliable, distributed, fairly fast, keeps everything in 
RAM, and expects to run continuously, but which there aren't going to be any 
ad hoc queries.

> It says it has "extremely fast real-time searches", but it also says 
> that the query language is "an add-on library"?

I guess it's extremely fast real-time searches *if* you know exactly what 
you're going to search for. Otherwise, it does a full table lock-and-scan 
without any optimization. And you have to maintain your own indexes, so 
after you insert the record into the table, you have to insert the key into 
each index as well.

The query language isn't the functions used to access the tables. It's sort 
of like saying "We have really fast loops and comparisons. List 
comprehensions are an add-on library."

> Reading the rest of the documentation just made my head hurt. I 
> particularly love how a "table" can be "set", "ordered set" or "bag", 
> but only the first and last option are explained.

That's exactly what I'm saying. You're expected to know that "set" and 
"ordered set" and "bag" are the three different types of ets tables, and go 
read the ets documentation to figure out what that is.

> A reference manual is no way to learn how to use a complex system.

In this case, the other choice is to go work for Ericson, it seems.

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

the real mistery of Erlang is the name:  at first I thought it was for 
"Ericsson Language".  But then I heard they were actually paying homage 
to one of the pioneers of traffic engineering and related telophony techs:

http://en.wikipedia.org/wiki/Agner_Krarup_Erlang

-- 
a game sig: http://tinyurl.com/d3rxz9

Post a reply to this message

On 10/03/2011 07:32 PM, nemesis wrote:
> the real mistery of Erlang is the name:

The real mystery of Erlang is that it was *banned* from Ericsson at one 
point...

Post a reply to this message

>> Erlang is partially functional, in that functions are apparently
>> first-class, and it's got single-assignment and immutable data
>> structures.
>
> Functions in C are first class. That doesn't mean it's functional.

In C it's not especially easy to write a half-line anonymous function 
and pass that as the argument to something. In fact, function pointers 
are probably the number #1 thing about C that I can't figure out how to 
work, but presumably this only affects me...

> The
> immutable data structures come from the single-assignment, except where
> the data structures aren't immutable, of which there are several.

I wasn't aware that there are any mutable data structures. Then again, I 
read an introductory design document, so...

> Well, that's the killer. To me, if every statement can have
> side-effects, then it's not a functional language.

Depends on your definition of "functional". Some people say that 
first-class functions is the definitive thing. Others claim it's 
referential transparency. A few people even claim it's the type system 
or the syntax...

> You have none of the benefits and all of the
> draw-backs of a functional language in Erlang.

Awesome.

>> The document I
>> read claims that it's considered beneficial to limit side-effects to
>> just a few places in the codebase,
>
> Except that message sending and message receiving are both side-effects
> as is spawning or linking another process, so that works pretty poorly.

The guy goes on to point out that by using gen_server, your code has no 
explicit message passing or process linking at all, and this is an 
extremely awesome thing because it means that your own code is free of 
side-effects.

> Plus, as I said, having the side-effects means you cannot (for example)
> use your own functions in guards.

Oh... my god.

> For some reason, the Erlang enthusiasts think the crappy syntax and the
> single-assignment procedures are intentional and beneficial properties
> of the language. Like any niche language, the fanboi's refuse to accept
> that any compromise was made during its design.

I'm not sure a language with multi-billion lines of code in 
mission-critical production applications can be considered "niche".

> Actually, in the documentation, I think there's a note on each built in
> function that says whether you can use it in a guard or not.

Fail.

>> One data point is scientific fact?
>
> It's a scientific fact that you can achieve nine 9's of uptime over the
> course of a decade with programs written in Erlang, yes. Pretty sure
> that's really, really hard to do with any other programming language.

Do we know that it isn't really, really hard to do in Erlang too? I 
mean, one system did it, but how much time, money and effort did they 
throw at the problem?

Now show that using something other than Erlang would have required 
/more/ time, money and effort, and we have something resembling a proven 
fact. ;-) Now repeat this for more than one system...

>> As I say, this doesn't really seem to be spelled out anywhere else
>> that I could find, and it seems a rather crucial detail.
>
> It's kind of in all the books and stuff. I think the PhD thesis is
> presumed required reading, if nothing else.

"The" PhD thesis?

>> (Quite how you tell the difference between an exception and a normal
>> result is left to the imagination...
>
> Well, an exception will start with EXIT, while a regular result is
> unlikely to, unless you specifically confuse things that way.

It just looks like the sort of thing I might do by mistake, spend 20 
minutes wondering why my application is acting screwy, and then have a 
little facepalm moment.

(E.g., one time I was building a Smalltalk application, and I wanted 
certain building blocks of the application to have names that would 
display on the screen. Obviously the name of a FirstOrderFilter doesn't 
change for each instance, so I added a class method called #name... All 
kinds of fun erupted after that. Eventually I discovered that the Class 
metaclass already /has/ a #name method, or something like that. Ouch!)

>> You mean that if message #5 doesn't arrive, message #6 is actually
>> guaranteed not to arrive?
>
> If message #5 didn't arrive because the receiving machine crashed, yes,
> message #6 isn't going to be received either. If message #5 didn't
> arrive because the network switch died, then #6 isn't going to be
> delivered either.

Presumably if the link dies and then comes back, or any other 
combination of circumstances, either the TCP link is still alive (in 
which case it resents #6 *and* #5), or the link has been broken (in 
which case neither message arrives).

> Yes, exactly. More precisely, you can't tell if the message was received
> and processed or not.

OK.

>> These people are /serious/ about reliability. o_O
>
> Well, yes, that's how you get 5 minutes of downtime over the course of
> 10 years.

Some people might refer to that as "luck". ;-) Apparently there are 
Windows 95 systems with this kind of uptime.

> Well, not quite. If the network between the machines dies, each machine
> is going to think the other crashed. This makes a mess.

"Ow, I don't care *what* universe you're from, that had to hurt!"

>> Thing is, I can open the REPL and type in a function, and then ask to
>> spawn that on a remote node. How does *that* work?!
>
> Hmmm. Actually, I think perhaps it *is* willing to ship functions
> around. The details escape me at the moment. Maybe you can send
> functions but not modules? I don't honestly remember.

Or maybe if you do this, it just tells you to sod off. I don't know...

>> I'll bet. No doubt somebody has written libraries for other languages
>> to speak the same wire protocol.
>
> I don't know. Probably, but I'm not sure why you would.

There are libraries that implement BEEP, even though nobody ever uses 
it. I'm *sure* there are libraries for a massively successful thing like 
Erlang. ;-)

>> I'm not actually sure why though.
>
> Reliability. ;-)

In what way is this more reliable?

>> You would think that a fully interconnected mesh would quickly exhaust
>> the supply of TCP ports too.
>
> Unless you have more than 50,000 machines, unlikely.

Hmm, yes. The number of links rises exponentially, not the number of 
links per node.

> One of the OTP functions shows how to authenticate and encrypt the
> connection en Erlang code, IIRC. That's also one of the tutorials.
>
> Actually, I had forgotten. There is a "cookie" you can set, and anyone
> connecting needs the same "cookie". So there is a minimal security
> level, not unlike the similar feature in X servers.

And the "cookie" is presumably sent unencrypted from node to node when 
they try to connect too.

BTW, how do you identify a remote node? Is it just by DNS name or IP 
address or something?

> What, you're believing fanboi's?

Doesn't everything? ;-)

>>> The two-versions limit is because the code itself isn't GCed. They
>>> document that they could have GCed the code, but that would have added
>>> lots of overhead for something relatively rare.
>>
>> Fair enough.
>
> Plus, you should be dealing with the occasional "process suddenly ends
> for no reason" exception robustly anyway. :-)

Layers within layers of crash recovery... :-}

> I found it annoying that there's no global roadmap. So you read about
> something and it says "If X happens, a message is sent to the error
> logger." WTF is the error logger? Where's the documentation. Then you
> track that down, and it's all talking about bits and pieces that are
> also not documented, etc, with the assumption that you have a bunch of
> people who wrote the error logger hanging around to tell you how to find
> where the errors got logged and such.

An error logger where you can't find the log? Oh, that's epic! :-D

Honestly, I thought only Haskell does this kind of stupidity...

(Yesterday, the new version of the Haskell Platform was released. Which 
is just as well, because the home page has been saying that the next 
release is due in "January 2011" for several months now.)

>> Escaped experiment, eh? ;-)
>
> That's exactly what it was. Except that by the time it escaped, there
> were millions of lines of production code written in it, so you couldn't
> even reasonably fix it.

Well, you can do syntax changes with a conversion program. But 
large-scale language changes? No, not really, no.

>> The advice presumably being that workers should only receive "real"
>> messages, and supervisors should only receive exceptions. Then there
>> can be no ambiguity. (Until the supervisor's supervisor asks it to do
>> something...)
>
> Well, more like, normal worker processes shouldn't communicate with
> messages that start with the atom "EXIT". It's not that hard.

It looks like the sort of mistake I would eventually make, and then 
spend a seriously long time trying to debug. Maybe being all upper-case 
is enough to prevent this, I don't know...

> Hermes gives you as many mailboxes as you want, with as many connections
> to each mailbox as you want, and you can send both connections and
> mailboxes in messages to other processes. So if you want hot-swap code,
> write your processes as being capable of packaging up its mailboxes and
> sending them to a new process.

OK.

>> So you're saying that all the "interesting" stuff like the SSL
>> implementation is actually just an off-the-shelf C library?
>
> It's possible.

What, you mean you don't *know*? Isn't this *documented* somewhere? ;-)

>> I never did understand what's so great about Tk. It looks horrid, it's
>> really hard to use, and it's almost impossible to get the layout you
>> want...
>
> Well, no, it's really easy to use, it's easy to get a good-looking
> layout, and it only looks horrid because it looked cutting-edge 10 years
> ago and nobody ever improved the look of it since then. (Altho they've
> worked on that more lately.)
>
> That's exactly why half a dozen non-Tcl languages actually went so far
> as to embed a Tcl interpreter into their code in order to be able to use
> Tk. I'm pretty sure it's not Tk that's at fault here. ;-)

That's like saying "C is a fantastic language, it's really easy to use, 
and that's why 90% of all software ever written uses it. I don't think C 
is at fault here."

>> Damn. If the actual switching is in hardware, that the heck do you
>> need the CPU for at all?
>
> For deciding what to connect. You pick up the phone? The 6800 connects
> your phone to the dial-tone generator. It's not sitting there generating
> the waveform in software. You push a button? The 6800 routes your line
> to the DTMF decoder, which signals the 6800 each time you let go of the
> tone. When you've dialed enough numbers, the 6800 figures out what line
> you're supposed to connect to and plugs you through.

That's seriously the only reason why it has a Turing-complete processing 
unit inside it?

Man, you would have thought QBASIC would be enough for _that_ level of 
functionality...

> I don't know. Go work for Erricson. ;-)

Worst FAQ answer *ever*! :-D

> It means the compiler is a function in the OTP libraries that you invoke
> in order to compile the code.

FWIW, I wish more languages were implemented like this...

(I'm just bitter because I spent yesterday trying to figure out how to 
get GHC to compile and optimise some source code, and then give me the 
AST. Which I managed, BTW, but trying to figure out WTF the AST actually 
means? Ugh!)

> You can look at the documentation for it to see what it does.

At this point, I don't hold out a lot of hope... o_O

> The other problem is that all the documentation assumes you've read the
> previous documentation and knows what they're talking about, but it
> doesn't actually give you pointers to any of the previous documentation.

I hate that. The Oracle documentation is like this too. Even the 
chapters in a single document refer to things out of sequence. Like, in 
chapter 1, it uses technical terms that aren't explained until chapter 
2, and chapter 2 uses things that were explained in chapter 1, but the 
explanation in chapter 1 doesn't make sense until you read chapter 3 
anyway. WHO WROTE THIS CRAP?!

Still, it could be worse. Consider the following quotation from one of 
the Haskell standard libraries:

http://www.haskell.org/ghc/docs/6.6/html/libraries/mtl/Control-Monad-Writer.html

"Inspired by the paper Functional Programming with Overloading and 
Higher-Order Polymorphism, Mark P Jones (http://www.cse.ogi.edu/~mpj/) 
Advanced School of Functional Programming, 1995."

(Notice that the URL no longer exists.)

I emphasise: /standard libraries/. Not even some exotic add-on library. 
One of the base libraries that comes with the compiler. And there's not 
one single line of human-written text on that page, beyond the reference 
to the paper where this stuff is actually explained.

> You'd like mnesia, tho. It's a lot like STM in some ways.

Except the part where there's at least 3 extensive yet readable 
technical introductions to what it does, why you'd want that, and how 
you use it, with full example source code and explanations. ;-)

Post a reply to this message

Invisible wrote:
> In C it's not especially easy to write a half-line anonymous function 

I didn't say it was convenient. I said it was first class.

>> immutable data structures come from the single-assignment, except where
>> the data structures aren't immutable, of which there are several.
> 
> I wasn't aware that there are any mutable data structures. Then again, I 
> read an introductory design document, so...

ETS and the process dictionary both spring to mind.

> Depends on your definition of "functional". Some people say that 
> first-class functions is the definitive thing. Others claim it's 
> referential transparency. A few people even claim it's the type system 
> or the syntax...

I can't think of any language nowadays that doesn't have first class 
functions. COBOL and FORTRAN, maybe, but that's about it.

I've never heard "functional" actually mean anything except "what you write 
is functions", as in, mapping inputs to outputs. Anything mutable makes it 
non-functional.

> The guy goes on to point out that by using gen_server, your code has no 
> explicit message passing or process linking at all, and this is an 
> extremely awesome thing because it means that your own code is free of 
> side-effects.

Yes. gen_server separates out the functional part from the non-functional 
part. But that doesn't give Erlang itself any benefits. No doubt functional 
programming is easier to debug. You just have to manually separate it out 
from the rest of the non-functional stuff.

>> Plus, as I said, having the side-effects means you cannot (for example)
>> use your own functions in guards.
> 
> Oh... my god.

Yes, exactly. Still think it's functional?

>> For some reason, the Erlang enthusiasts think the crappy syntax and the
>> single-assignment procedures are intentional and beneficial properties
>> of the language. Like any niche language, the fanboi's refuse to accept
>> that any compromise was made during its design.
> 
> I'm not sure a language with multi-billion lines of code in 
> mission-critical production applications can be considered "niche".

It's niche, as much as formula-1 race cars are niche cars in spite of being 
famous and wildly expensive. ERlang's niche is telecom programs that run for 
decades without crashing.

>> It's a scientific fact that you can achieve nine 9's of uptime over the
>> course of a decade with programs written in Erlang, yes. Pretty sure
>> that's really, really hard to do with any other programming language.
> 
> Do we know that it isn't really, really hard to do in Erlang too? I 
> mean, one system did it, but how much time, money and effort did they 
> throw at the problem?

Well, let's rephrase that. It's certainly easier in Erlang than it would be 
in straight C. :-)

> Now show that using something other than Erlang would have required 
> /more/ time, money and effort, and we have something resembling a proven 
> fact. ;-) Now repeat this for more than one system...

Look up the history of NIL. Then look up the history of Hermes. Now you have 
three similarly-organized systems that all show you can run long-term 
mixed-developer systems reliably by using these techniques.

Do you think Ericson had *not* been trying to come up with this system 
before they wrote millions of lines of code in it? Do you think they 
wouldn't have started using a different system if Erlang didn't give them 
the benefits they expected?

> 
>>> As I say, this doesn't really seem to be spelled out anywhere else
>>> that I could find, and it seems a rather crucial detail.
>>
>> It's kind of in all the books and stuff. I think the PhD thesis is
>> presumed required reading, if nothing else.
> 
> "The" PhD thesis?

Armstrong's PhD thesis.

>>> (Quite how you tell the difference between an exception and a normal
>>> result is left to the imagination...
>>
>> Well, an exception will start with EXIT, while a regular result is
>> unlikely to, unless you specifically confuse things that way.
> 
> It just looks like the sort of thing I might do by mistake, spend 20 
> minutes wondering why my application is acting screwy, and then have a 
> little facepalm moment.

I didn't use Erlang very much, and it never occurred to me to use an atom in 
all upper case, let alone one that says 'EXIT'.

> (E.g., one time I was building a Smalltalk application, and I wanted 
> certain building blocks of the application to have names that would 
> display on the screen. Obviously the name of a FirstOrderFilter doesn't 
> change for each instance, so I added a class method called #name... All 
> kinds of fun erupted after that. Eventually I discovered that the Class 
> metaclass already /has/ a #name method, or something like that. Ouch!)

Name clash of methods? Sure. Name clash of values you're returning against 
the three or four already-defined values? No. Treat it as a reserved word.

This is exactly why so many messages returned start with "ok".

> Presumably if the link dies and then comes back, or any other 
> combination of circumstances, either the TCP link is still alive (in 
> which case it resents #6 *and* #5), or the link has been broken (in 
> which case neither message arrives).

Yes.

> Some people might refer to that as "luck". ;-) Apparently there are 
> Windows 95 systems with this kind of uptime.

No there aren't. All the 16-bit Windows machines had a seconds counter that 
wrapped after 42 days and crashed the computer.

>>> I'll bet. No doubt somebody has written libraries for other languages
>>> to speak the same wire protocol.
>>
>> I don't know. Probably, but I'm not sure why you would.
> 
> There are libraries that implement BEEP, even though nobody ever uses 
> it. 

Actually, I'm rather surprised at the number of people who use 
syslog-reliable (which runs over BEEP) let alone BEEP itself. What makes you 
think nobody uses BEEP?

>>> I'm not actually sure why though.
>> Reliability. ;-)
> In what way is this more reliable?

If you lose a connection to a node, you can ask the other nodes if they lost 
a connection too, for one thing.  And if A connects to B, then B passes a 
connection to C to A, A needs a connection to C anyway. You don't want to 
wait till that point for A to find out it is firewalled from C.

> And the "cookie" is presumably sent unencrypted from node to node when 
> they try to connect too.

No, the cookie doesn't get sent at all.

> BTW, how do you identify a remote node? Is it just by DNS name or IP 
> address or something?

http://www.erlang.org/doc/reference_manual/distributed.html#id80719

The documentation *is* online.

>> Plus, you should be dealing with the occasional "process suddenly ends
>> for no reason" exception robustly anyway. :-)
> 
> Layers within layers of crash recovery... :-}

Incidentally, NIL worked it where you declared which processes were a group, 
and you could kill them off or roll them over as a group. So you didn't have 
the problem of one module being upgraded while another module isn't.

> An error logger where you can't find the log? Oh, that's epic! :-D

I'm sure you can find it if you can figure out which module it talks to. But 
it's like .NET, where debugging output goes to the configured output 
methods, and if you haven't configured them, it disappears. If you're used 
to (say) C's assert() macro, finding out you have to configure where assert 
messages go on a run-by-run basis is rather disconcerting if you didn't even 
think to look.

> It looks like the sort of mistake I would eventually make, and then 
> spend a seriously long time trying to debug. Maybe being all upper-case 
> is enough to prevent this, I don't know...

Why do you make mistakes like that?  How often do you write code and use 
reserved works as variable names?

> What, you mean you don't *know*? Isn't this *documented* somewhere? ;-)

It's in the source code. Welcome to open source!

> That's like saying "C is a fantastic language, it's really easy to use, 
> and that's why 90% of all software ever written uses it. I don't think C 
> is at fault here."

No, because there really wasn't much of an alternative to C for many years. 
People tried many alternatives, and they all sucked so bad they never really 
got out of the lab. So clearly C is doing something right.

> That's seriously the only reason why it has a Turing-complete processing 
> unit inside it?

I don't know what else it does. Billing records, managing the configuration 
changes, etc. The point is it's not switching individual voice packets, any 
more than your CPU is getting involved in deciding which ethernet packets 
going past are addressed to the machine the ethernet card is plugged into.

>> It means the compiler is a function in the OTP libraries that you invoke
>> in order to compile the code.
> 
> FWIW, I wish more languages were implemented like this...

Well, that's the next version of .NET.  I think it's cool. I wish people who 
weren't using Microsoft could use it as easily.

> (I'm just bitter because I spent yesterday trying to figure out how to 
> get GHC to compile and optimise some source code, and then give me the 
> AST. Which I managed, BTW, but trying to figure out WTF the AST actually 
> means? Ugh!)

Erlang can hand you the AST for you to run macro preprocessign on at compile 
time. (That's how records work, for example.) You think figuring out the AST 
when you have a *statically* typed language is hard?

> Except the part where there's at least 3 extensive yet readable 
> technical introductions to what it does, why you'd want that, and how 
> you use it, with full example source code and explanations. ;-)

http://www.erlang.org/doc/apps/mnesia/users_guide.html

Were you maybe reading the reference manual instead of the user's guide?

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

Invisible <voi### [at] devnull> wrote:
> - There are many functional programming languages. (Haskell, OCaml, 
> Clean, and for some reason people keep calling Lisp "functional" too.) 

  Why did you not mention Lisp? It's the first, and still considered by
many to be *The* functional language.

> However, Erlang is *the only* one that could be considered "commercially 
> successful", as far as I can tell.

  How about Lisp?

> Suffice it to say, from what little I could discover, I didn't like what 
> I was seeing. Like most commercially successful languages, Erlang is 
> obtuse, complex, ugly and kludgy. Much like C, Java or anything else 
> wildly popular.

  Can you name a programming language that isn't? (Besides Lisp.)

  And don't say "Haskell". I can understand a Java program much more
easily than a Haskell one. So much for obtuse, complex and ugly.

-- 
                                                          - Warp

Post a reply to this message

On 11/03/2011 05:53 PM, Warp wrote:
> Invisible<voi### [at] devnull>  wrote:
>> - There are many functional programming languages. (Haskell, OCaml,
>> Clean, and for some reason people keep calling Lisp "functional" too.)
>
>    Why did you not mention Lisp?

Ummm... I *did*? It's right there in the quote.

> It's the first, and still considered by
> many to be *The* functional language.

I can't figure out why. Functions are first-class in Lisp, but beyond 
that there's nothing functional about it. (Unless you think "it uses 
lists" has something to do with being functional...)

>> However, Erlang is *the only* one that could be considered "commercially
>> successful", as far as I can tell.
>
>    How about Lisp?

I wasn't aware that there are any multi-million line mission-critical 
production-grade systems written in Lisp. I suppose there might be 
somewhere. I'm certainly not aware of any well-known commercial entities 
telling everybody that they should use Lisp.

>> Suffice it to say, from what little I could discover, I didn't like what
>> I was seeing. Like most commercially successful languages, Erlang is
>> obtuse, complex, ugly and kludgy. Much like C, Java or anything else
>> wildly popular.
>
>    Can you name a programming language that isn't? (Besides Lisp.)

No, I'd say Lisp is kludgy too.

>    And don't say "Haskell". I can understand a Java program much more
> easily than a Haskell one. So much for obtuse, complex and ugly.

There are people who describe mathematical equations as being obtuse, 
complex and ugly. And there are others who describe them as being 
sublimely beautiful, even transcendental. Wanna guess which side I'm on?

Beauty is of course in the eye of the beholder, so I won't argue about 
this further.

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

>> Depends on your definition of "functional". Some people say that
>> first-class functions is the definitive thing. Others claim it's
>> referential transparency. A few people even claim it's the type system
>> or the syntax...
>
> I can't think of any language nowadays that doesn't have first class
> functions. COBOL and FORTRAN, maybe, but that's about it.

Java? Eiffel?

> Yes. gen_server separates out the functional part from the
> non-functional part. But that doesn't give Erlang itself any benefits.
> No doubt functional programming is easier to debug. You just have to
> manually separate it out from the rest of the non-functional stuff.

Indeed. It lets you write in a functional style, but doesn't force you 
to. I suppose some people will claim that's "pragmatic".

>> Oh... my god.
>
> Yes, exactly. Still think it's functional?

Personally, no...

>> I'm not sure a language with multi-billion lines of code in
>> mission-critical production applications can be considered "niche".
>
> It's niche, as much as formula-1 race cars are niche cars in spite of
> being famous and wildly expensive. ERlang's niche is telecom programs
> that run for decades without crashing.

Sure. But people often reverse the word "niche" as a kind of polite way 
of saying "hopelessly unsuccessful". ;-)

>>> It's a scientific fact that you can achieve nine 9's of uptime over the
>>> course of a decade with programs written in Erlang, yes. Pretty sure
>>> that's really, really hard to do with any other programming language.
>>
>> Do we know that it isn't really, really hard to do in Erlang too?
>
> Well, let's rephrase that. It's certainly easier in Erlang than it would
> be in straight C. :-)

People claim that Unix (or maybe specificaly Linux) is insanely 
reliable, and that's almost all pure C.

> Do you think Ericson had *not* been trying to come up with this system
> before they wrote millions of lines of code in it? Do you think they
> wouldn't have started using a different system if Erlang didn't give
> them the benefits they expected?

I'm sure Ericsson things that Erlang is great. That's not "scientific" 
necessarily though. ;-) I'm not seriously disagreeing with you, just the 
fanboys.

>> "The" PhD thesis?
>
> Armstrong's PhD thesis.

I wonder if that's the document I just read, or a different one...

>> It just looks like the sort of thing I might do by mistake, spend 20
>> minutes wondering why my application is acting screwy, and then have a
>> little facepalm moment.
>
> I didn't use Erlang very much, and it never occurred to me to use an
> atom in all upper case, let alone one that says 'EXIT'.

> This is exactly why so many messages returned start with "ok".

So many, BUT NOT ALL. It seems to vary inconsistently. Sometimes there's 
an atom to tell you whether it was OK or not, and sometimes there's only 
an atom to tell you when it failed.

>> Some people might refer to that as "luck". ;-) Apparently there are
>> Windows 95 systems with this kind of uptime.
>
> No there aren't. All the 16-bit Windows machines had a seconds counter
> that wrapped after 42 days and crashed the computer.

Got a reference for that?

>>>> I'm not actually sure why though.
>>> Reliability. ;-)
>> In what way is this more reliable?
>
> If you lose a connection to a node, you can ask the other nodes if they
> lost a connection too, for one thing.

Oh, I see.

> And if A connects to B, then B
> passes a connection to C to A, A needs a connection to C anyway.

What makes you think that?

>> And the "cookie" is presumably sent unencrypted from node to node when
>> they try to connect too.
>
> No, the cookie doesn't get sent at all.

So how do you compare it then?

>> BTW, how do you identify a remote node? Is it just by DNS name or IP
>> address or something?
>
> http://www.erlang.org/doc/reference_manual/distributed.html#id80719
>
> The documentation *is* online.

Right. So it's DNS name, plus a user-defined atom (presumably so you can 
run several nodes on a single machine).

>> It looks like the sort of mistake I would eventually make, and then
>> spend a seriously long time trying to debug. Maybe being all
>> upper-case is enough to prevent this, I don't know...
>
> Why do you make mistakes like that? How often do you write code and use
> reserved works as variable names?

More often than you'd think. For example, I might try to use "if" for 
"input file" and "of" for "output file". Shame both of those are 
reserved words in Haskell. But - fortunately - this causes a 
compile-time error in Haskell. You say "treat special atoms as reserved 
words", but if they really were reserved words, you'd get a compile-time 
error informing you of your mistake, rather than mysterious run-time 
behaviour which possibly only shows up under unusual conditions.

>> What, you mean you don't *know*? Isn't this *documented* somewhere? ;-)
>
> It's in the source code. Welcome to open source!

EPIC FAILURE. >_<

>> That's like saying "C is a fantastic language, it's really easy to
>> use, and that's why 90% of all software ever written uses it. I don't
>> think C is at fault here."
>
> No, because there really wasn't much of an alternative to C for many
> years. People tried many alternatives, and they all sucked so bad they
> never really got out of the lab. So clearly C is doing something right.

That's the actual reason? I've often wondered about this, but it was a 
long time ago when all this stuff presumably happened...

>> That's seriously the only reason why it has a Turing-complete
>> processing unit inside it?
>
> I don't know what else it does. Billing records, managing the
> configuration changes, etc. The point is it's not switching individual
> voice packets, any more than your CPU is getting involved in deciding
> which ethernet packets going past are addressed to the machine the
> ethernet card is plugged into.

Wait - the CPU on my PC *doesn't* process the incomming packets??

> Erlang can hand you the AST for you to run macro preprocessign on at
> compile time. (That's how records work, for example.) You think figuring
> out the AST when you have a *statically* typed language is hard?

Strictly, what I'm looking at isn't the parse tree of the Haskell source 
code. I'm looking at the internal data structures used by the Haskell 
optimiser. And it's not so much that the data is complex, it's that most 
of it is complex, and it's difficult to guess where the hell the 
function for examining it is defined. And the API just plain isn't very 
documented. I get the impression that the inside of the compiler is just 
very complex, and not especially tidy, which is surprising.

> http://www.erlang.org/doc/apps/mnesia/users_guide.html
>
> Were you maybe reading the reference manual instead of the user's guide?

Yeah, possibly.

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Orchid XP v8 wrote:
>>> Depends on your definition of "functional". Some people say that
>>> first-class functions is the definitive thing. Others claim it's
>>> referential transparency. A few people even claim it's the type system
>>> or the syntax...
>>
>> I can't think of any language nowadays that doesn't have first class
>> functions. COBOL and FORTRAN, maybe, but that's about it.
> 
> Java? Eiffel?

Well, alright. You have to wrap them up in an anonymous class.

> Sure. But people often reverse the word "niche" as a kind of polite way 
> of saying "hopelessly unsuccessful". ;-)

I can't help people don't know english. :-)

> People claim that Unix (or maybe specificaly Linux) is insanely 
> reliable, and that's almost all pure C.

I've never heard anyone claim that.  I've heard Linux people bash Windows, 
and I've heard people brag about how reliable Linux is, but it's certainly 
not reliable enough that it'll run ten years worth of upgrades without a 
reboot or loss of service, unless you go to something like a Tandem.

> So many, BUT NOT ALL. It seems to vary inconsistently. Sometimes there's 
> an atom to tell you whether it was OK or not, and sometimes there's only 
> an atom to tell you when it failed.

In practice, in those routines (like catch) that return the value or an atom 
if it failed, people return an atom tagging the success as well. Or they 
return a different type like a list.

>>> Some people might refer to that as "luck". ;-) Apparently there are
>>> Windows 95 systems with this kind of uptime.
>>
>> No there aren't. All the 16-bit Windows machines had a seconds counter
>> that wrapped after 42 days and crashed the computer.
> 
> Got a reference for that?

I'm sorry your computer can't get through to Google. I hope your 
connectivity gets restored soon.

http://news.cnet.com/2100-1040-222391.html

>> And if A connects to B, then B
>> passes a connection to C to A, A needs a connection to C anyway.

 > What makes you think that?

Because B may pass to A a connection to a mailbox on C.

>>> And the "cookie" is presumably sent unencrypted from node to node when
>>> they try to connect too.
>>
>> No, the cookie doesn't get sent at all.
> 
> So how do you compare it then?

Oh, I see what you're asking. I assume they're smart enough not to send 
passwords in the clear to machines that haven't proven they already know the 
password.

>> Why do you make mistakes like that? How often do you write code and use
>> reserved works as variable names?
> 
> More often than you'd think. 

Wow.

>> No, because there really wasn't much of an alternative to C for many
>> years. People tried many alternatives, and they all sucked so bad they
>> never really got out of the lab. So clearly C is doing something right.
> 
> That's the actual reason? I've often wondered about this, but it was a 
> long time ago when all this stuff presumably happened...

C was designed to be a portable that was *not* abstracted from the machine. 
Therefore, it got ported a lot, often as the first HLL ported to a new 
architecture.  The other languages only got ported to architectures being 
used for that particular type of computing.

When people tried to make better languages than C, they tended to pile so 
much extra stuff into it that would make it appropriate for both 
machine-level stuff and high-level applications that it turned into a monster.

> Wait - the CPU on my PC *doesn't* process the incomming packets??

Not packets not destined for your machine, no. WTF do you think a MAC 
address is for?

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

Orchid XP v8 wrote:
> I can't figure out why. Functions are first-class in Lisp, but beyond 
> that there's nothing functional about it. 

Early versions of LISP were functional. It was for a long time the *only* 
popular language you could write anything that looked like functional code in.

Which is the functional language: FORTRAN, C, COBOL, or LISP?

> I wasn't aware that there are any multi-million line mission-critical 
> production-grade systems written in Lisp. I suppose there might be 
> somewhere. I'm certainly not aware of any well-known commercial entities 
> telling everybody that they should use Lisp.

Yahoo was originally all in LISP before it got sold, including the store. 
Several large video games are written in LISP. I think the right answer is 
that more people use LISP than you think, but they don't advertise it 
because it's a competitive advantage.

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

>> Sure. But people often reverse the word "niche" as a kind of polite
>> way of saying "hopelessly unsuccessful". ;-)
>
> I can't help people don't know english. :-)

I love the way people use "legacy" to mean "that ancient hunk of junk 
that we somehow ended up still having to support".

>> People claim that Unix (or maybe specificaly Linux) is insanely
>> reliable, and that's almost all pure C.
>
> I've never heard anyone claim that. I've heard Linux people bash
> Windows, and I've heard people brag about how reliable Linux is,

When Linux first became popular, this was *the reason* to switch to 
Linux. Everything is so reliable and works perfectly without crashing 
every 12 seconds. (Never mind that they don't have drivers for your disk 
controller yet, never mind your graphics card... who cares about that?)

> but
> it's certainly not reliable enough that it'll run ten years worth of
> upgrades without a reboot or loss of service, unless you go to something
> like a Tandem.

I can't say I've tried.

>>>> Some people might refer to that as "luck". ;-) Apparently there are
>>>> Windows 95 systems with this kind of uptime.
>>>
>>> No there aren't. All the 16-bit Windows machines had a seconds counter
>>> that wrapped after 42 days and crashed the computer.
>>
>> Got a reference for that?
>
> I'm sorry your computer can't get through to Google. I hope your
> connectivity gets restored soon.
>
> http://news.cnet.com/2100-1040-222391.html

...which links to http://support.microsoft.com/kb/q216641/ which says 
that they fixed this problem 10 years ago.

>>> And if A connects to B, then B
>>> passes a connection to C to A, A needs a connection to C anyway.
>
>> What makes you think that?
>
> Because B may pass to A a connection to a mailbox on C.

Sorry, I misread your sentence.

> Oh, I see what you're asking. I assume they're smart enough not to send
> passwords in the clear to machines that haven't proven they already know
> the password.

I'd like to believe that... *shuffle*

>>> Why do you make mistakes like that? How often do you write code and use
>>> reserved works as variable names?
>>
>> More often than you'd think.
>
> Wow.

Hey, we all make mistakes...

(Like the time I discovered that "default" is a reserved word in 
Haskell. I had to look at the language spec to find out what it's 
actually for, since nobody ever actually uses it.)

> C was designed to be a portable that was *not* abstracted from the
> machine. Therefore, it got ported a lot, often as the first HLL ported
> to a new architecture. The other languages only got ported to
> architectures being used for that particular type of computing.
>
> When people tried to make better languages than C, they tended to pile
> so much extra stuff into it that would make it appropriate for both
> machine-level stuff and high-level applications that it turned into a
> monster.

I guess the answer is only obvious in retrospect...

>> Wait - the CPU on my PC *doesn't* process the incomming packets??
>
> Not packets not destined for your machine, no. WTF do you think a MAC
> address is for?

For telling you which machine a packet is addressed to? You still have 
to *read* the MAC address to find out if that's you, surely.

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Orchid XP v8 wrote:
> (Like the time I discovered that "default" is a reserved word in 
> Haskell. I had to look at the language spec to find out what it's 
> actually for, since nobody ever actually uses it.)

I must admit, that's the one reserved word I don't infrequently use for a 
variable by mistake. I got over it after the first five or ten times, tho.

> For telling you which machine a packet is addressed to? You still have 
> to *read* the MAC address to find out if that's you, surely.

Who is "you" in that sentence? Your CPU? No.

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

> Which is the functional language: FORTRAN, C, COBOL, or LISP?

That's like saying "which is the high-level language: machine code, 
assembly language, or C?"

Oh, wait...

> I think the right
> answer is that more people use LISP than you think, but they don't
> advertise it because it's a competitive advantage.

That's a pretty glowing recommendation for Lisp. Assuming it's actually 
true of course. ;-)

Post a reply to this message

Invisible wrote:
>> Which is the functional language: FORTRAN, C, COBOL, or LISP?
> 
> That's like saying "which is the high-level language: machine code, 
> assembly language, or C?"

More like "which is the portable one?"

At the time it was written, LISP was the only language that *could* be 
programmed in a functional way. You can't write functional COBOL or 
functional FORTRAN.

Later, constructs were added to let to LISP you assign values and such.

> That's a pretty glowing recommendation for Lisp. Assuming it's actually 
> true of course. ;-)

I think you'd be surprised at some of the languages used in closed-source 
projects.

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

Darren New <dne### [at] sanrrcom> wrote:
> Invisible wrote:
> >> Which is the functional language: FORTRAN, C, COBOL, or LISP?
> >
> > That's like saying "which is the high-level language: machine code,
> > assembly language, or C?"
>
> More like "which is the portable one?"
>
> At the time it was written, LISP was the only language that *could* be
> programmed in a functional way. You can't write functional COBOL or
> functional FORTRAN.
>
> Later, constructs were added to let to LISP you assign values and such.
>
> > That's a pretty glowing recommendation for Lisp. Assuming it's actually
> > true of course. ;-)
>
> I think you'd be surprised at some of the languages used in closed-source
> projects.

yep, a Lisp-like language was used by Naughty Dog to script AI in the Crash
Bandicoot games and even as far as the more recent Uncharted games by same
company for PS3.  OTOH, it was built as a culture within the company previous to
developments of python and Lua as more usable alternatives for the task...

this was in the 90's and early 2000's, when Lisp kind of made a resurgence
thanks to Paul Graham and his successful account of how Lisp allowed him beat
the averages.

Lisp is pretty much dead nowadays, even the flagship company for it, ITA
Software, today employs much java, python and C++... Lisp is mostly legacy.

even still, I'd say it's still used more than haskell... ;)

Post a reply to this message

nemesis <nam### [at] gmailcom> wrote:
> yep, a Lisp-like language was used by Naughty Dog to script AI in the Crash
> Bandicoot games and even as far as the more recent Uncharted games by same
> company for PS3.  OTOH, it was built as a culture within the company previous to
> developments of python and Lua as more usable alternatives for the task...

  I have never actually been in a large game company, but it is my
understanding that using all kinds of interpreted (or sometimes bytecompiled)
scripting languages is rather usual in games.

  The core game engine as well as many of the middleware libraries (such
as the physics engine) are, AFAIK, usually written in C++ (with many C
libraries thrown in), but many of the higher-level functionalities are
often written in whatever scripting language the team likes (such as
Lua, Python, or a custom language of the particular game engine). It makes
it easier to change things (such as AI behavior) without having to recompile
the project. Also "scripted events" can literally have a double meaning:
Besides following a pre-written "script", they are often implemented using
a scripting language.

-- 
                                                          - Warp

Post a reply to this message

On 3/15/2011 7:42, Warp wrote:
>    The core game engine as well as many of the middleware libraries (such
> as the physics engine) are, AFAIK, usually written in C++ (with many C
> libraries thrown in), but many of the higher-level functionalities are
> often written in whatever scripting language the team likes

Unreal engine is exactly like that, yes. All the collision stuff and 
animation sequencing and things are done in system languages, while the code 
for what to do (like, "Spawn here, then use the RUN animation until you get 
within X distance of the player") are in Unreal Script.

It's pretty cool. All the documentation is online.

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message

>> The core game engine as well as many of the middleware libraries (such
>> as the physics engine) are, AFAIK, usually written in C++ (with many C
>> libraries thrown in), but many of the higher-level functionalities are
>> often written in whatever scripting language the team likes
>
> Unreal engine is exactly like that, yes. All the collision stuff and
> animation sequencing and things are done in system languages, while the
> code for what to do (like, "Spawn here, then use the RUN animation until
> you get within X distance of the player") are in Unreal Script.

Valve say they did something similar with [at least] the particle 
effects in Team Fortress 2. They made it text-scriptable, so that 
animators can design new particle effects for their weapons and models 
and stuff without having to call a C++ programmer to write the animation 
code.

> It's pretty cool. All the documentation is online.

Uh... Valve makes their Source SDK *available* online, but it can only 
be considered *documented* for sufficiently small values of "documentation".

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

On 3/16/2011 3:11, Orchid XP v8 wrote:
> Uh... Valve makes their Source SDK *available* online, but it can only be
> considered *documented* for sufficiently small values of "documentation".

That's the difference between buying a commercial engine designed for 
writing your own games, and buying someone's finished game and them throwing 
in the SDK for free.

Speaking of Erlang...

-- 
Darren New, San Diego CA, USA (PST)
  "How did he die?"   "He got shot in the hand."
     "That was fatal?"
          "He was holding a live grenade at the time."

Post a reply to this message