It has been said that Haskell (and languages like it) will forever 
remain an interesting curiosity until the day when large, real-world, 
*useful* applications exist written in it. Only then will anybody 
seriously take any notice.

Erlang has Wings 3D. You don't have to know or care about Erlang to use 
Wings. It's impressive in its own right, not because it's Erlang. There 
is something to be said for the argument that no programming language 
can really be taken seriously until it has at least one real-world 
application that people actually use for real stuff.

Somebody somewhere wrote that nobody really gave a damn about Ruby until 
Ruby on Rails came along. Now of course *everybody* knows about Ruby. 
And apparently every "web framework" must be compared to Rails, because 
it is obviously the best framework that has ever existed in the history 
of mankind.

I have literally no idea what a "web framework" is. I don't know 
anything about Ruby, and I have no idea what Rails actually does. What I 
*do* know is that there's been a lot of buzz in the Haskell community 
lately about several Haskell web frameworks under aggressive development 
right now. Apparently a few people are of the opinion that one of these 
might finally be the Killer App to put Haskell on the map. So I thought 
I'd take a look...

(As it turns out, since the connection between web browser and web 
server is untyped, this becomes an interesting instance of Darren's 
"static typing is pointless" problem.)



I started with the implausibly-named (and unpronounceable) "Yesod" 
framework.

The very first thing I noticed is that it's *documented*. Understand 
this: Most Haskell projects consist of half a dozen source code files. 
When you look at the project page, you see a list of all the data 
structures and functions that it exports, and nothing more. IF YOU'RE 
LUCKY, there might be some human-written exposition saying what each 
thing on the list does. Maybe even a tiny snippet of example code. And 
that is all.

Part of the problem is that the Haskell build system has hard-wired 
support for Haddock, an API reference document generator similar to 
Doxygen, JavaDoc, and a bazillion others. Which means that every package 
on Hackage.org has an API reference (regardless of whether the author 
intended this). If you're lucky, the author will have written some 
documentation to go with the API reference. But only if you're lucky.

An API reference - even a *good* one - is no substitute for a 
comprehensive *explanation* of what the hell is actually going on. But 
Haddock doesn't support that kind of thing, and Hackage has no way to 
include such documentation even if you happen to have written it. The 
best you can do is host your own website (at your own expense) and 
include the URL in the package description.

Yesod *has* its own website, and some pretty extensive documentation. 
It's not perfect, but it's vastly better than 99% of Hackage. So I set 
about reading all the documentation they provide:

http://www.yesodweb.com/book/

The results of my investigation were... interesting.

First of all, this is no monolithic block of code. It's a vast 
arrangement of many separate libraries, and clearly some effort has been 
directed towards making it easy to use a different building block for 
this function or that function if you so desire. Having said that, 
clearly the "default" combination of components is going to be the one 
that's most thoroughly tested and takes the least amount of work to use, 
out of the box.

Briefly summarising what I've learned, it seems Yesod offers a medium 
level of abstraction. There have been other projects that do much more 
work for you, at the expense of forcing you to design your application 
in a specific way. Yesod seems to be a somewhat lower-level offering, 
giving you more flexible options as a consequence, but also making you 
work a bit harder.

[For example, there was an experimental system where you wrote 
user/computer interactions as transactions in a special monad, so it 
looked just like a regular CLI program, but under the hood the framework 
auto-generated HTML forms, session cookies, ran a non-relational 
database engine to hold user state, handled the back-button by rolling 
back transactions automatically, and all kinds of freaky stuff. It's 
very impressive that such a thing can be done, but obviously it does 
utterly force you to follow the framework designer's vision of The One 
True Way that a web application should work.]

The way Yesod works is interesting. By default, it uses its own internal 
web server named "Warp". It's written in 100% Haskell, and it's 
supposedly extremely fast (hence the name). Like so many things, Warp 
wasn't actually developed as part of Yesod, it was just what they chose 
to have as their default component.

I gather from the blurb, though, that it's supposed to be possible to 
run your application as a CGI binary instead. [Although the 
documentation warns you to never, ever do this, because forking a new 
process for every connection would be far too unperformant.] Fast-CGI is 
also supported, and presumably much faster if you're already running 
Apache or something. SCGI also works. You're even supposed to be able to 
use WebKit to make "native desktop applications".

The next interesting thing is how content works. On a typical web 
server, you have static pages and other resources, and then you have CGI 
*scripts*, or possibly various kinds of templates that look like HTML, 
but they go through a preprocessor which inserts external data into 
certain places within the template.

Yesod does it differently. Templates are Haskell source code. They can 
be in separate files or inline in your main program, but either way, 
every template gets compiled into executable machine code. This means 
that template processing is lightning fast. It also means that every 
single damned time you change anything in any template anywhere, you 
have to recompile that template, and relink the entire executable.

By default, the Warp web server is used. Since this is written in 
Haskell, the end result is that the web server, application logic, 
presentation templates and static files are all compiled into one giant 
binary executable. When you run this, it opens a TCP port and starts 
listening for HTTP requests.

It also means that any time anything in any template changes, that 
template must be recompiled, and then the entire program relinked. And 
then you have to shut down the running server and fire up the newly 
compiled on.

All of which seems... a bit strange, to me. Then again, if you want to 
use Apache, you can build your application for Fast-CGI instead. Then 
only the application logic and presentation templates get linked into a 
giant binary blob, and you don't have to restart the entire server to 
update. It still seems like a rather heavyweight approach.

The templates make use of Haskell's "quasi-quoting" feature. This 
basically allows you to embed arbitrary strings into Haskell source 
code, provided you write a parser. But it also means that those strings 
can refer to any Haskell things that are currently in-scope. For 
example, you can write a Haskell loop, and the loop body can be a 
template which refers to the loop variable.

One of the features the manual makes a big deal of is "typesafe URLs". 
Essentially, instead of using strings to refer to things, you create 
data structures which represent valid URLs. And I *don't* mean that you 
have a data structure which represents the parsed pieces of a URL. I 
mean you have a dummy data type called "HomePage", and then a data type 
called "UserProfile" with a user ID field, and then a "Topic" with a 
topic ID field. And then you tell Yesod how to convert "UserProfile 42" 
into a URL, and how to convert a URL into a UserProfile 42 again.

That done, all your templates refer to UserProfile 42. The compiler is 
therefore about to statically check, at compile-time, that every 
generated URL is valid. You can never accidentally misspell a URL 
(misspelling a data type is a compile-time error), you can never forget 
a component of a pathname (there *are* no pathnames). If you want to 
change the mapping from logical resources to physical URLs, you change 
it in one single place. You don't have to search the whole codebase and 
change dozens of URLs. About the worst thing that can happen is that you 
say UserProfile 138, and there *isn't* a user number 138 at the moment. 
(And in that case, you can return a HTTP 404 saying "no such user", 
rather than just a generic "file not found".)

In a similar way, the mapping from URLs to "handlers" (the code that 
actually decides what data to return for a given URL) is declarative, 
not imperative. You specify which URL maps to what data type, and what 
handler. If URLs overlap, that is a compile-time error. Any URL you 
don't provide a mapping for is HTTP 404.

Similar to all this, Yesod uses the type system to distinguish between 
raw text which needs to be escaped before being inserted into HTML / CSS 
/ JS (with the correct escaping rules for each one), and text which is 
already in such a format (and must /not/ be escaped, because that would 
mangle it). So there's one syntax for inserting stuff into a web page, 
and the type system detects whether it needs to be escaped first.

(Unfortunately, there's a different syntax for inserting URLs. Plus 
there's half a dozen slightly different templating languages, and the 
differences are poorly explained. Still, "whamlet". I thought only XKCD 
talked about WHAM! anymore...)

Amusingly, there's a system to run a template, take an MD5 hash of the 
inputs to it, and save the output to disk so it can be cached on the 
server side for faster delivery, and on the client side for reduced 
server load.

Yesod provides other goodies. For example, it has a "widget" concept. 
The idea appears to be such that you can design, say, a date picker, and 
make that a library. You can then reuse this wherever you want it. In 
particular, you can have two date pickers on the same page.

Why can't you do that already? Well, problem #1: a widget like this 
probably consists of an HTML form plus some CSS to style it plus some 
JavaScript to run it. These three elements go in different parts of the 
page. Yesod's widget concept handles taking lots of widgets and putting 
all the CSS in one place, all the JS in another place, etc. Problem #2: 
name collisions. Again, Yesod lets you generate unique IDs to get around 
that. (E.g., for form names, variable names, etc.)

Personally, I think Yesod's widgets are a nice idea, but a tad minimal. 
They probably need more functionality to really be useful.

Widgets aren't just for forms, of course. You can use them for navbars 
or news panels or anything you like. And then Yesod has a dedicated 
forms module, which is meant to allow you to take existing forms and 
transparently glue them together to make bigger forms. Again, I'm 
unconvinced of how well it really works, but I admire the idea.

There's also a "session" feature which lets you store key/value pairs in 
a cookie. The interesting part is that the cookie is encrypted with a 
key only the server knows (so clients can't see what's in it), and 
signed (so clients can't change the contents). It adds overhead, but I 
don't think I've ever seen it this easy before...

Then there's the "persistent" framework, where you write some Haskell 
data structure definitions, and Yesod generates SQL to create the 
database, and boilerplate code for converting SQL data into Haskell data 
and vice versa. [Actually, it supports "no SQL" databases too. And comes 
with a default "in-memory database" for test purposes.] Now if you try 
to put a string as the user ID value, you get a compile-time error, 
because that field *should* be an integer.

The interesting question, of course, is "what happens if the schema 
changes?" It seems Yesod is *assuming* that the schema will never change 
unless Yesod changes it. (I presume I don't need to point out how 
jaw-droppingly flawed such an assumption is.) Yesod provides a feature 
to let you do a "migration". And by that, I mean that Yesod will load up 
the current database schema, compare to what your compiled application 
is expecting the schema to be, and then issue a bunch of ALTER TABLE 
commands to make the database match what the application expects.

If you put a migration statement at the start of your application, then 
each time your application starts up, it will change the database to 
match the schema it expects. That means if you alter your application so 
the schema changes, when you recompile and run the new version, the 
database is automatically updated before the application starts running.

While this *does* take care of part of the problem, it's by no means a 
perfect solution. Immediate problems include:

- There are situations where Yesod can't correct the database 
automatically. (The main one being where a field is renamed. Yesod has 
no idea what to rename, just by looking at it.) Still, at least the 
application will halt at the migration step, rather than running for a 
few days and *then* crashing in the middle of a rarely-used query.

- Just blindly munging the schema as you see fit is fine for small toy 
applications. I wouldn't dare do such a thing in a production 
environment, however. Perhaps the extra fields in the DB but not in 
Haskell are used by somebody else? We really shouldn't just go and 
delete them just like that...

- ...which brings us to the fundamental problem with the whole design 
here. It fundamentally assumes that the database is *only* for this one 
application. Now it seems to have somehow been lost to history, but I 
have C. J. Date on my bookshelf, and he tells me that part of the 
definition of a "database" is that it is a store of data accessed by 
MORE THEN ONE APPLICATION. So no single application should just blindly 
alter the schema as it sees fit. That's why SQL databases have things 
like views; it lets different programs see different parts of the 
database, without having to rejigger every application every time the 
data changes, nor alter the data when any application changes. That's 
one of the fundamental reasons to use a proper database rather than a 
plain flat file!

Still, given the current crazy for "no-SQL databases" (i.e., "we don't 
need no stinking coherent theory of operation!"), I guess this point is 
lost on most designers...

Given this assumption, it comes as no surprise that Yesod hasn't even 
considered the possibility that a schema might change at run-time. But 
then again, if it does, what can you actually do about it? Either the 
tables and fields you need are present, or they aren't. If they aren't, 
that's pretty much *got* to be a fatal run-time error.

About the only schema change that looks potentially survivable is if a 
field changes type. If Haskell is only moving that data from A to B, 
then arguably if Haskell /didn't/ check its type, this change could be 
survivable. Then again, I'm struggling to think of a scenario where the 
type wouldn't matter and just calling it "text" or "raw binary" wouldn't 
be OK.



Having digested all of that, I went and took a look at Happstack:

http://happstack.com/docs/crashcourse/index.html

(The "Haskell Application Server Stack".)

A surprising amount of the details were similar in concept or downright 
*identical* to Yesod. (In some cases, it's actually using the same 
components for various jobs.) For example, typesafe URLs make another 
appearence, although now with imperative routing rather than declarative.

While Yesod defaults to using the Shakespare series of template 
languages, Happstack appears to be quite happy for you to use 
Shakespare, BlazeHtml, HString, Heist or several others.

Shakespare uses fancy syntax resembling normal HTML / CSS / JS and 
compiler trickery to convert this into regular Haskell code. BlazeHtml 
uses regular Haskell code in the first place. Heist, on the other hand, 
has templates which are just regular HTML files, and it searches them 
for markers where dynamic content should be inserted. In other words, 
templates are just regular HTML that any web designer would understand, 
and they can be changed without recompiling (or restarting) your 
application server.

[Incidentally, Heist itself is part of "Snap", yet *another* Haskell web 
framework, which I have yet to properly research...]

Then there's JMacro, which allows you to embed JavaScript directly in 
your Haskell source file, and have it syntax-checked at compile-time. 
(I.e., the final JavaScript sent to the web browser will never have 
syntax errors; these errors are caught at Haskell compile-time.) Plus of 
course, it lets you splice chunks of JS together, generate unique 
variable or function names, insert Haskell data into JS, and so on.

As an extra, JMacro allows Haskell-style anonymous function definitions. 
Quite why the hell that's useful I'm not sure. Oh, and also ML-style 
functions, which are much more wordy. WTF?



So there you have it. It seems there's quite a few web frameworks out 
there now. And I just can't help noticing that *all* of the example code 
I've looked at is really very, very long and difficult to follow, 
considering the trivial amount of functionality it actually 
implements... o_O

Post a reply to this message

Invisible escreveu:
> It has been said that Haskell (and languages like it) will forever 
> remain an interesting curiosity until the day when large, real-world, 
> *useful* applications exist written in it. Only then will anybody 
> seriously take any notice.
> 
> Erlang has Wings 3D.

That's not quite a large app, but I digress.

> I have literally no idea what a "web framework" is. I don't know 
> anything about Ruby, and I have no idea what Rails actually does.

We all know how much you enjoy knowing nothing and still brag about it. :)

> (As it turns out, since the connection between web browser and web 
> server is untyped, this becomes an interesting instance of Darren's 
> "static typing is pointless" problem.)

I concur.

> http://www.yesodweb.com/book/
> 
> The results of my investigation were... interesting.
> 
> The next interesting thing is how content works. On a typical web 
> server, you have static pages and other resources, and then you have CGI 
> *scripts*, or possibly various kinds of templates that look like HTML, 
> but they go through a preprocessor which inserts external data into 
> certain places within the template.
> 
> Yesod does it differently. Templates are Haskell source code. They can 
> be in separate files or inline in your main program, but either way, 
> every template gets compiled into executable machine code. This means 
> that template processing is lightning fast. It also means that every 
> single damned time you change anything in any template anywhere, you 
> have to recompile that template, and relink the entire executable.
> 
> By default, the Warp web server is used. Since this is written in 
> Haskell, the end result is that the web server, application logic, 
> presentation templates and static files are all compiled into one giant 
> binary executable. When you run this, it opens a TCP port and starts 
> listening for HTTP requests.

Just imagine letting thousands of different users compiling their own 
"pages" into your single blob exec.

> It also means that any time anything in any template changes, that 
> template must be recompiled, and then the entire program relinked. And 
> then you have to shut down the running server and fire up the newly 
> compiled on.

yeah, well, guess at some point in our lives we all did the pure C 
approach to web programming, just to better understand how it all works. 
  Blazing fast, but slow as hell workflow...

> All of which seems... a bit strange, to me. Then again, if you want to 
> use Apache, you can build your application for Fast-CGI instead. Then 
> only the application logic and presentation templates get linked into a 
> giant binary blob, and you don't have to restart the entire server to 
> update. It still seems like a rather heavyweight approach.

still better.

> The templates make use of Haskell's "quasi-quoting" feature.

quasi-quoting is a TM from Lisp industries... :)

> One of the features the manual makes a big deal of is "typesafe URLs". 
> Essentially, instead of using strings to refer to things, you create 
> data structures which represent valid URLs.
> And I *don't* mean that you 
> have a data structure which represents the parsed pieces of a URL. I 
> mean you have a dummy data type called "HomePage", and then a data type 
> called "UserProfile" with a user ID field, and then a "Topic" with a 
> topic ID field. And then you tell Yesod how to convert "UserProfile 42" 
> into a URL, and how to convert a URL into a UserProfile 42 again.

that is a fine feature.  But I still prefer regex. :)

> Yesod provides other goodies. For example, it has a "widget" concept. 

oh, how original.

> There's also a "session" feature which lets you store key/value pairs in 
> a cookie.

OHMY, php devs should sue them!

> The interesting part is that the cookie is encrypted with a 
> key only the server knows (so clients can't see what's in it), and 
> signed (so clients can't change the contents). It adds overhead, but I 
> don't think I've ever seen it this easy before...

done.

> - ...which brings us to the fundamental problem with the whole design 
> here. It fundamentally assumes that the database is *only* for this one 
> application. Now it seems to have somehow been lost to history, but I 
> have C. J. Date on my bookshelf, and he tells me that part of the 
> definition of a "database" is that it is a store of data accessed by 
> MORE THEN ONE APPLICATION. So no single application should just blindly 
> alter the schema as it sees fit. That's why SQL databases have things 
> like views; it lets different programs see different parts of the 
> database, without having to rejigger every application every time the 
> data changes, nor alter the data when any application changes. That's 
> one of the fundamental reasons to use a proper database rather than a 
> plain flat file!
> 
> Still, given the current crazy for "no-SQL databases" (i.e., "we don't 
> need no stinking coherent theory of operation!"), I guess this point is 
> lost on most designers...
> 
> Given this assumption, it comes as no surprise that Yesod hasn't even 
> considered the possibility that a schema might change at run-time. But 
> then again, if it does, what can you actually do about it? Either the 
> tables and fields you need are present, or they aren't. If they aren't, 
> that's pretty much *got* to be a fatal run-time error.

every framework has their own way of doing things.  You either comply or 
go on your own.  Don't battle the tool, use it.

anyway, most frameworks have the same problems with databases.

> Then there's JMacro, which allows you to embed JavaScript directly in 
> your Haskell source file, and have it syntax-checked at compile-time. 
> (I.e., the final JavaScript sent to the web browser will never have 
> syntax errors; these errors are caught at Haskell compile-time.)

the amazing thing about these typesafe nuts are that they are so 
determined to the point of even basically writing syntax checks for 
foreign languages! o_0

but they are too lazy, why not also provide type safety for javascript? 
  Do not stop at syntax checks, go down to the guts of type checks as 
well! :D

> So there you have it. It seems there's quite a few web frameworks out 
> there now. And I just can't help noticing that *all* of the example code 
> I've looked at is really very, very long and difficult to follow, 
> considering the trivial amount of functionality it actually 
> implements... o_O

ruby on rails is simpler. :)

Post a reply to this message

>> Erlang has Wings 3D.
>
> That's not quite a large app, but I digress.

Not "large", but certainly widely used, and not just by Erlang fans.

>> I have literally no idea what a "web framework" is. I don't know
>> anything about Ruby, and I have no idea what Rails actually does.
>
> We all know how much you enjoy knowing nothing and still brag about it. :)

I'm just pointing out that I have nothing else to compare it to, that's 
all. Not *everybody* has used Rails before. :-P

>> By default, the Warp web server is used. Since this is written in
>> Haskell, the end result is that the web server, application logic,
>> presentation templates and static files are all compiled into one
>> giant binary executable. When you run this, it opens a TCP port and
>> starts listening for HTTP requests.
>
> Just imagine letting thousands of different users compiling their own
> "pages" into your single blob exec.

Yeah, I'm guessing the result is extremely fast page loading times, and 
a system administration nightmare behind the scenes...

Then again, presumably the answer is to split the server into smaller 
components, and have them talk to each other via SCGI or something.

>> It also means that any time anything in any template changes, that
>> template must be recompiled, and then the entire program relinked. And
>> then you have to shut down the running server and fire up the newly
>> compiled on.
>
> yeah, well, guess at some point in our lives we all did the pure C
> approach to web programming, just to better understand how it all works.
> Blazing fast, but slow as hell workflow...

Except that in C, you will waste most of your time tracking down silly 
typos that the compiler isn't smart enough to protect you from. In 
Haskell, you "just" have to worry about the sheer inefficiency of trying 
to repeatedly tweak code that has to go through a compiler.

[Well, no... you *can* actually run it interpretted. I'm guessing that 
would be spectacularly slow.]

>> It still seems like a rather heavyweight approach.
>
> still better.

Sure. But perhaps not "best". For example, the person who writes the 
HTML templates needs to know Haskell. Not just the guy who writes the 
application logic.

>> The templates make use of Haskell's "quasi-quoting" feature.
>
> quasi-quoting is a TM from Lisp industries... :)

I doubt that what Haskell calls "quasi-quoting" actually works the same 
way as Lisp. Much like C++ functors are nothing like what Haskell calls 
functors...

>> One of the features the manual makes a big deal of is "typesafe URLs".
>> Essentially, instead of using strings to refer to things, you create
>> data structures which represent valid URLs.
>> And I *don't* mean that you have a data structure which represents the
>> parsed pieces of a URL. I mean you have a dummy data type called
>> "HomePage", and then a data type called "UserProfile" with a user ID
>> field, and then a "Topic" with a topic ID field. And then you tell
>> Yesod how to convert "UserProfile 42" into a URL, and how to convert a
>> URL into a UserProfile 42 again.
>
> that is a fine feature. But I still prefer regex. :)

And I still hate regex with a passion. :-P

Being more objective... Haskell can warn you at compile-time if your 
patterns overlap. Or do real parsing on data fields. And the order in 
which you specify patterns is guaranteed not to matter. And the compiler 
can optimise out repeated tests at compile-time. And...

>> Yesod provides other goodies. For example, it has a "widget" concept.
>
> oh, how original.

I know, right? Who would have thought of that?

>> There's also a "session" feature which lets you store key/value pairs
>> in a cookie.
>
> OHMY, php devs should sue them!

I don't know, most systems I've seen [i.e., not many] make it 
unnecessarily difficult to do something as trivial as set or get a 
cookie value.

>> Given this assumption, it comes as no surprise that Yesod hasn't even
>> considered the possibility that a schema might change at run-time. But
>> then again, if it does, what can you actually do about it? Either the
>> tables and fields you need are present, or they aren't. If they
>> aren't, that's pretty much *got* to be a fatal run-time error.
>
> every framework has their own way of doing things. You either comply or
> go on your own. Don't battle the tool, use it.
>
> anyway, most frameworks have the same problems with databases.

Yeah, figures. After all, a database is only for running websites, 
right? It's not as if at some point somebody in the back office is going 
to want to know how many users logged in today or something like that...

>> Then there's JMacro, which allows you to embed JavaScript directly in
>> your Haskell source file, and have it syntax-checked at compile-time.
>> (I.e., the final JavaScript sent to the web browser will never have
>> syntax errors; these errors are caught at Haskell compile-time.)
>
> the amazing thing about these typesafe nuts are that they are so
> determined to the point of even basically writing syntax checks for
> foreign languages! o_0

What, like it's hard? JavaScript is a fairly small language.

> but they are too lazy, why not also provide type safety for javascript?
> Do not stop at syntax checks, go down to the guts of type checks as
> well! :D

Not possible. Determining whether an arbitrary program is safe-typed is 
undecinable in the general case. (Type systems make it decidable by 
arbitrarily rejecting certain safe-typed programs because they are not 
well-typed, and well-typeness is [designed to be] decidable.)

>> So there you have it. It seems there's quite a few web frameworks out
>> there now. And I just can't help noticing that *all* of the example
>> code I've looked at is really very, very long and difficult to follow,
>> considering the trivial amount of functionality it actually
>> implements... o_O
>
> ruby on rails is simpler. :)

Perhaps. But is it *better*? ;-)

-- 
http://blog.orphi.me.uk/
http://www.zazzle.com/MathematicalOrchid*

Post a reply to this message

Orchid XP v8 escreveu:
>> ruby on rails is simpler. :)
> 
> Perhaps. But is it *better*? ;-)

I'll let you find out. :)

-- 
a game sig: http://tinyurl.com/d3rxz9

Post a reply to this message

On 11/17/2011 6:21, Invisible wrote:
> Erlang has Wings 3D.

Errr, no. Erlang has the north american telephone switching system. :-)

> this becomes an interesting instance of Darren's "static typing is
> pointless" problem.)

Indeed, this is exactly the point I was making.

> an API reference document generator

And that is why documentation sucks these days. Those aren't documentation 
generators. Those are documentation extractors. If you don't put the 
documentation in before you run it, you don't get any documentation out.

> Yesod does it differently. Templates are Haskell source code. They can be in
> separate files or inline in your main program, but either way, every
> template gets compiled into executable machine code.

This is how most systems that compile templates work, including Java and 
ASP.NET.

> damned time you change anything in any template anywhere, you have to
> recompile that template, and relink the entire executable.

Except that because Java and .NET don't do type erasure, you don't have to 
actually relink anything but the one script. And ASP.NET even notices the 
executable changed and reloads it (or notices the source changed and 
recompiles it and reloads it), letting existing transactions finish and new 
transactions power up the new code.

> By default, the Warp web server is used. Since this is written in Haskell,
> the end result is that the web server, application logic, presentation
> templates and static files are all compiled into one giant binary
> executable. When you run this, it opens a TCP port and starts listening for
> HTTP requests.

Very convenient for deployment, you must exist. If you're writing an 
application (like, a wiki manager say), distributing it this way is an 
excellent way of doing it.

> It also means that any time anything in any template changes, that template
> must be recompiled, and then the entire program relinked. And then you have
> to shut down the running server and fire up the newly compiled on.

Generally not a problem, really.

> All of which seems... a bit strange, to me. Then again, if you want to use
> Apache, you can build your application for Fast-CGI instead. Then only the
> application logic and presentation templates get linked into a giant binary
> blob, and you don't have to restart the entire server to update. It still
> seems like a rather heavyweight approach.

If you need better than that, you're probably doing it wrong anyway. Or you 
should be using something like Erlang instead.

> worst thing that can happen is that you say UserProfile 138, and there
> *isn't* a user number 138 at the moment.

Or the user dicks with the URL and bypasses something in your code you 
thought would be enforced.

> Similar to all this, Yesod uses the type system to distinguish between raw
> text which needs to be escaped before being inserted into HTML / CSS / JS
> (with the correct escaping rules for each one), and text which is already in
> such a format (and must /not/ be escaped, because that would mangle it). So
> there's one syntax for inserting stuff into a web page, and the type system
> detects whether it needs to be escaped first.

*That* is the sort of thing static typing is good for. That's the problem 
hungarian naming conventions were trying to solve for languages that 
couldn't distinguish "integer number of elements in the array" and "integer 
value of blue component of that pixel".

> Amusingly, there's a system to run a template, take an MD5 hash of the
> inputs to it, and save the output to disk so it can be cached on the server
> side for faster delivery, and on the client side for reduced server load.

I was doing that many, many years ago. Before HTML had <FORM> tags. Indeed, 
if you look up "REST" interfaces, that's basically what they're about.

> Why can't you do that already?

Because you can? :-) Heck, "widget" is even the standard term for it. Heck, 
sophisticated systems like ASP.NET even have compile-time attributes on the 
widgets so you can use tools to lay them out on the screen at compile time. 
(E.g., your calendar widget would declare how big it is and what type it 
manipulates so your IDE can do the right thing with displaying it while 
you're designing the web page and dragging stuff around.)

> widget concept handles taking lots of widgets and putting all the CSS in one
> place, all the JS in another place, etc.

And GWT takes Java, compiles it into javascript and CSS and HTML, compresses 
each of those, and bundles it into one big file. Same idea.

> Again, I'm unconvinced of how well
> it really works, but I admire the idea.

Given that pretty much everyone in the world does it this way, yah, it seems 
it's an idea that's here to stay. :-)

> There's also a "session" feature which lets you store key/value pairs in a
> cookie. The interesting part is that the cookie is encrypted with a key only
> the server knows (so clients can't see what's in it), and signed (so clients
> can't change the contents). It adds overhead, but I don't think I've ever
> seen it this easy before...

ASP.NET.  It's exactly that. You just tag the variables you want to store 
there, and when the user posts back, the framework fills in those variables, 
then gives you notifications as if it's a desktop app and the user just 
typed new values in.

> The interesting question, of course, is "what happens if the schema
> changes?" It seems Yesod is *assuming* that the schema will never change
> unless Yesod changes it. (I presume I don't need to point out how
> jaw-droppingly flawed such an assumption is.)

That's the other half of the "static typing isn't that useful" rant, you 
see. That's exactly what I was talking about there, if you go back and read 
again. :-)

> - Just blindly munging the schema as you see fit is fine for small toy
> applications. I wouldn't dare do such a thing in a production environment,
> however. Perhaps the extra fields in the DB but not in Haskell are used by
> somebody else? We really shouldn't just go and delete them just like that...

Exactly. Almost nobody who builds these things ever thinks as the database 
as soemthing other than a convenient place to store their data. All the 
"NoSQL" people who don't need ACID all assume they are the only application 
using the database.  As soon as sales and manufacturing and shipping and 
warranty returns and customer billing are all updating the same database, 
you probably don't want to be randomly changing the schema.

> Still, given the current crazy for "no-SQL databases" (i.e., "we don't need
> no stinking coherent theory of operation!"), I guess this point is lost on
> most designers...

Everyone thinks they're google. :-)  Funny thing is, even the nosql data 
storage at google has schemas, because you couldn't even manage it 
otherwise. "Why is my database suddenly 3x the size it should be?"

-- 
Darren New, San Diego CA, USA (PST)
   People tell me I am the counter-example.

Post a reply to this message

On 18/11/2011 04:07 AM, Darren New wrote:
> On 11/17/2011 6:21, Invisible wrote:
>> Erlang has Wings 3D.
>
> Errr, no. Erlang has the north american telephone switching system. :-)

Well, yeah, and Haskell has defence contractors, investment bankers, 
cryptographers and IC designers using it. AND NOBODY GIVES A DAMN. :-P

>> this becomes an interesting instance of Darren's "static typing is
>> pointless" problem.)
>
> Indeed, this is exactly the point I was making.

And hence, I thought you might be interested to see how a couple of 
tools approach [or ignore] the problem. I know I was...

>> an API reference document generator
>
> And that is why documentation sucks these days. Those aren't
> documentation generators. Those are documentation extractors. If you
> don't put the documentation in before you run it, you don't get any
> documentation out.

On the plus side, this is Haskell. Sometimes the type signature is 
enough to actually tell you WHAT THE CODE DOES. That's a pretty amazing 
property, which few other languages have.

On the other hand, sometimes the type signature DOES NOT tell you what 
the code does. And then not having any *real* documentation utterly sucks.

Also, Haskellers have this bizarre perversion where some people think 
that describing the axioms that a structure satisfies as the same thing 
as explaining WHAT IT DOES, WHY THAT'S USEFUL or HOW YOU USE IT. :-P

>> Yesod does it differently. Templates are Haskell source code. They can
>> be in
>> separate files or inline in your main program, but either way, every
>> template gets compiled into executable machine code.
>
> This is how most systems that compile templates work, including Java and
> ASP.NET.

I've never heard of a system which actually /compiles/ templates.

>> damned time you change anything in any template anywhere, you have to
>> recompile that template, and relink the entire executable.
>
> Except that because Java and .NET don't do type erasure, you don't have
> to actually relink anything but the one script.

I'm not seeing how type erasure is in any way related to the ability or 
inability to dynamically load new code.

> And ASP.NET even notices
> the executable changed and reloads it (or notices the source changed and
> recompiles it and reloads it), letting existing transactions finish and
> new transactions power up the new code.

Well, yeah, the Yesod scaffolder sets things up so that if you change 
any code, and then restart the server, it recompiles anything first.

Alternatively, you can run it in an interpreter. But I don't know what 
performance would be like...

>> By default, the Warp web server is used. Since this is written in
>> Haskell,
>> the end result is that the web server, application logic, presentation
>> templates and static files are all compiled into one giant binary
>> executable. When you run this, it opens a TCP port and starts
>> listening for
>> HTTP requests.
>
> Very convenient for deployment, you must exist. If you're writing an
> application (like, a wiki manager say), distributing it this way is an
> excellent way of doing it.

"You must exist"? What, I post, therefore I am? ;-)

But yeah, it's probably quite nice for final deployment. I'm unsure what 
it's like for development.

>> It also means that any time anything in any template changes, that
>> template
>> must be recompiled, and then the entire program relinked. And then you
>> have
>> to shut down the running server and fire up the newly compiled on.
>
> Generally not a problem, really.

Well, assuming you're not hacking up changes on your production box. 
(Why would you *ever* do that?)

>> All of which seems... a bit strange, to me. Then again, if you want to
>> use
>> Apache, you can build your application for Fast-CGI instead. Then only
>> the
>> application logic and presentation templates get linked into a giant
>> binary
>> blob, and you don't have to restart the entire server to update. It still
>> seems like a rather heavyweight approach.
>
> If you need better than that, you're probably doing it wrong anyway. Or
> you should be using something like Erlang instead.

I'm not sure I follow.

>> worst thing that can happen is that you say UserProfile 138, and there
>> *isn't* a user number 138 at the moment.
>
> Or the user dicks with the URL and bypasses something in your code you
> thought would be enforced.

No, that's kind of the point. Each URL is checked against the list of 
valid URL structures. If it matches one, it gets passed off to that 
chunk of code. If it matches none, it's 404. (That includes checking 
that, say, the user ID *is* a valid number, and *isn't* negative, and so 
forth.)

>> Similar to all this, Yesod uses the type system to distinguish between
>> raw
>> text which needs to be escaped before being inserted into HTML / CSS / JS
>> (with the correct escaping rules for each one), and text which is
>> already in
>> such a format (and must /not/ be escaped, because that would mangle
>> it). So
>> there's one syntax for inserting stuff into a web page, and the type
>> system
>> detects whether it needs to be escaped first.
>
> *That* is the sort of thing static typing is good for. That's the
> problem hungarian naming conventions were trying to solve for languages
> that couldn't distinguish "integer number of elements in the array" and
> "integer value of blue component of that pixel".

What it does mean is that you can take a string, insert it into a small 
HTML template, insert the small HTML template into a larger HTML 
template, and bolt on some other templates. The original string gets 
escaped before insertion. The HTML does /not/ get escaped when inserted 
into the rest. And so forth. All of which you /could/ keep track of 
manually, but it would be really easy to get it wrong...

>> Amusingly, there's a system to run a template, take an MD5 hash of the
>> inputs to it, and save the output to disk so it can be cached on the
>> server
>> side for faster delivery, and on the client side for reduced server load.
>
> I was doing that many, many years ago. Before HTML had <FORM> tags.
> Indeed, if you look up "REST" interfaces, that's basically what they're
> about.

Yesod claims to be "RESTful". I have literally no idea WTF that actually 
means. (As best as I can tell, it's a philosophy, not something more 
concrete.)

>> widget concept handles taking lots of widgets and putting all the CSS
>> in one
>> place, all the JS in another place, etc.
>
> And GWT takes Java, compiles it into javascript and CSS and HTML,
> compresses each of those, and bundles it into one big file. Same idea.

Of course, desktop interfaces have always worked like this. It's news to 
me that anyone has ever successfully implemented this for a web 
interface, however.

>> Again, I'm unconvinced of how well
>> it really works, but I admire the idea.
>
> Given that pretty much everyone in the world does it this way, yah, it
> seems it's an idea that's here to stay. :-)

Again, I've never seen it done on the web.

>> There's also a "session" feature which lets you store key/value pairs
>> in a
>> cookie. The interesting part is that the cookie is encrypted with a
>> key only
>> the server knows (so clients can't see what's in it), and signed (so
>> clients
>> can't change the contents). It adds overhead, but I don't think I've ever
>> seen it this easy before...
>
> ASP.NET. It's exactly that. You just tag the variables you want to store
> there, and when the user posts back, the framework fills in those
> variables, then gives you notifications as if it's a desktop app and the
> user just typed new values in.

1. Every system I've seen makes it unnecessarily hard to use cookies.

2. I haven't seen a system which actually applies cryptography to the 
problem.

>> The interesting question, of course, is "what happens if the schema
>> changes?" It seems Yesod is *assuming* that the schema will never change
>> unless Yesod changes it. (I presume I don't need to point out how
>> jaw-droppingly flawed such an assumption is.)
>
> That's the other half of the "static typing isn't that useful" rant, you
> see. That's exactly what I was talking about there, if you go back and
> read again. :-)

...which is why this is an interesting point to discuss.

Yesod has gone one step further than simply ignoring the problem. But 
only one small step.

> Exactly. Almost nobody who builds these things ever thinks as the
> database as soemthing other than a convenient place to store their data.
> All the "NoSQL" people who don't need ACID all assume they are the only
> application using the database. As soon as sales and manufacturing and
> shipping and warranty returns and customer billing are all updating the
> same database, you probably don't want to be randomly changing the schema.

This is precisely my point, and it seems to be lost on most people.

If you want to store just the data your application uses, go ahead. 
Knock yourself out. You can just dump it all into a flat file. That 
works, right? And you can structure that file in whatever way is most 
efficient for your particular application.

What's that? Now you want somebody *else* to be able to access it too? 
Oh dear, that's just too bad. :-P

This is exactly what a real database engine allows. It lets you query 
data in /arbitrary/ ways, not just the ways that the application 
designer considered. It lets multiple applications use the same data at 
once. It lets you change the physical storage of the data, without 
changing its logical appearance. It lets different applications see 
different logical views of the data, so you can change one app without 
needing to change them all. And so on and so forth.

And yet, everybody seems to think that a database is just a flat file, 
but better...

>> Still, given the current crazy for "no-SQL databases" (i.e., "we don't
>> need
>> no stinking coherent theory of operation!"), I guess this point is
>> lost on
>> most designers...
>
> Everyone thinks they're google. :-) Funny thing is, even the nosql data
> storage at google has schemas, because you couldn't even manage it
> otherwise. "Why is my database suddenly 3x the size it should be?"

I'm definitely not Google. o_O

It did amuse me how Yesod says that "by being non-relational, we gain 
more flexibility and the ability to avoid the overhead of table joins". 
Well, you know what? Table joins have overhead in time. But denormalised 
data has overhead in space. And redundancy. And you know what else? 
Every half-sane RDBMS optimises the **** out of table joins. You can 
even cache the result on disk if you want. And you know what? Then you 
have denormalised data, without any of the runtime overhead, but the 
database MANAGES THE REDUNDANCY FOR YOU! Whereas the alternative is to 
store all your data denormalised, and manage the redundancy yourself, 
manually. Oh, I hope you don't make a mistake. Because, let's face it, 
there's lots of opportunities to get it wrong...

Yes, relational databases are just unnecessary overhead. :-P

Post a reply to this message

One thing I don't understand... memcached. Um, WHY?

As best as I can tell, it's a cache that sits between the database and 
the application. Except that... any half-decent database *already* 
caches query results. WTF?

Post a reply to this message

Le 2011-11-18 04:43, Invisible a écrit :
> On 18/11/2011 04:07 AM, Darren New wrote:
>> On 11/17/2011 6:21, Invisible wrote:
>>> Erlang has Wings 3D.
>>
>> Errr, no. Erlang has the north american telephone switching system. :-)
>
> Well, yeah, and Haskell has defence contractors, investment bankers,
> cryptographers and IC designers using it. AND NOBODY GIVES A DAMN. :-P
>

Aren't defense contractors all using ADA?  Or did that change over the 
last 10 years?


-- 
/*Francois Labreque*/#local a=x+y;#local b=x+a;#local c=a+b;#macro P(F//
/*    flabreque    */L)polygon{5,F,F+z,L+z,L,F pigment{rgb 9}}#end union
/*        @        */{P(0,a)P(a,b)P(b,c)P(2*a,2*b)P(2*b,b+c)P(b+c,<2,3>)
/*   gmail.com     */}camera{orthographic location<6,1.25,-6>look_at a }

Post a reply to this message

>>>> Erlang has Wings 3D.
>>>
>>> Errr, no. Erlang has the north american telephone switching system. :-)
>>
>> Well, yeah, and Haskell has defence contractors, investment bankers,
>> cryptographers and IC designers using it. AND NOBODY GIVES A DAMN. :-P
>>
>
> Aren't defense contractors all using ADA? Or did that change over the
> last 10 years?

"All" might be an exaggeration. But you're right, there's probably more 
of them using Ada than there are using Haskell...

Post a reply to this message

On 17/11/2011 02:21 PM, Invisible wrote:

> I started with the implausibly-named (and unpronounceable) "Yesod"
> framework.

The good part: It's very easy to install. Assuming you already have 
Haskell installed, it takes one command to download and compile all the 
source code. And it even works on Windows.

The bad part, arguably, is that it downloads, compiles and installs 85 
individual Haskell packages. Which, obviously, takes a little while. (!) 
And then, when you run the scaffolder to build a skeleton site and try 
to run it, it downloads and installs even *more* packages! By the end of 
it, using the Sqlite DB, I had 97 packages installed. If you want to use 
more backends, obviously you end up with even more packages.

Also interesting: The scaffolder executable is 7MB in size. An empty 
site using the "tiny" scaffold is 22MB, and one using the "sqlite" 
scaffold is 34MB. This for a site that serves one single HTML page. 
(Static linking FTW!)

[You would have thought, given that they went to all the bother of 
providing a scaffolder tool, they could make it produce a really 
good-looking default site. But no; it looks like 1993 in there.]

> Having digested all of that, I went and took a look at Happstack

By contrast, Happstack consists of 16 packages. There is no handy 
scaffolder, but a simple "hello world" application is 3 lines of code. 
Compiling it, I get a 10MB application, which delivers a single piece of 
text. (Not HTML, just text.)

Post a reply to this message