Darren New <dne### [at] sanrrcom> wrote:
> http://www.codeproject.com/KB/cross-platform/BenchmarkCppVsDotNet.aspx

> It should be noted that the compact framework doesn't do nearly as much work 
> optimizing stuff that the desktop systems do, and that's well known. The GC 
> is known to suck, if nothing else, but overall a lot of stuff is interpreted 
> without caching and such rather than use all the tricks that go into modern 
> VM-based systems.

  I have always detested interlanguage benchmark comparisons (even in
situations where my pet language comes on top). The reason is that making
a fair comparison is extremely difficult and what can be considered "fair"
is very up to interpretation.

  How do you implement the same program in two different programming
languages so that it's fair for both?

  One solution would be to make both implementations as identical as
possible to each other, by always using the closest thing in both languages
in every situation. However, this can be very unfair because it very often
happens that this solution favors one of the languages over the other,
introducing a bad bias.

  It may be that the best way to implement the task is different in the
languages, eg. because of technical reasons. For example, memory allocation
sucks in terms of speed in most C++ compilers, while in many other languages
it's very fast. OTOH, C++ often beats those other languages in raw number
crunching that does not involve constant memory allocation. If the aim is
to implement a task in the most efficient way in both languages, these
implementations may often need to use quite different approaches.

  Another possible approach is to take an expert, competent programmer for
each of the languages and tell them to implement the task as efficiently
as they can.

  This can be a much fairer approach, but might suffer from the
implementations being detached from actual, practical programs and might
in fact not represent the *typical* (but still competent) average solution
in that language. In other words, if this becomes a competition on who comes
up with the fastest solution, the end result might be a mess of hacker
optimization that has little to do with practical programming. It might be
fast, but it usually won't be very good code (in terms of maintainability,
readability, etc.)

-- 
                                                          - Warp

Post a reply to this message