On 6/21/2011 11:01, Warp wrote:
>    I have always detested interlanguage benchmark comparisons (even in
> situations where my pet language comes on top). The reason is that making
> a fair comparison is extremely difficult and what can be considered "fair"
> is very up to interpretation.

Agreed. That's why I thought all the comments on the article about how it 
was more a comparison of the standard libraries used were missing the point 
of what he was benchmarking. Clearly there's a difference between "the 
fastest you can get out of a particular language/compiler/environment 
regardless of the cost of the implementation" and "the speed it goes when 
you do everything normally."

Clearly his measuring of P/Invoke overhead shows he understands how these 
things work.

But it *is* interesting on a general level. I've heard "Ruby is slow" and 
I've heard "Python is slow", but I haven't seen any hard numbers at all 
there. How slow? 10x as slow? 100x as slow? Etc. Saying "you'll need 20% 
more servers in order to finish the project in 35% less time" might be a 
good trade-off, might be a bad trade-off. But without even a vague idea, 
it's impossible to say.

The problem is that the vocal people who discuss these things in public are 
generally people who are trying to prove a point, so it's never treated as a 
value-per-unit-effort comparison and instead is always treated as "maybe my 
jet flies into a mountain if you do a garbage-collect at the wrong time, oh 
noes!"

-- 
Darren New, San Diego CA, USA (PST)
   "Coding without comments is like
    driving without turn signals."

Post a reply to this message