On Tue, 08 Aug 2006 11:46:01 +0200, Warp <war### [at] tagpovrayorg> wrote:
> Alain <ele### [at] netscapenet> wrote:
>> OK then. So, as long as your results STAY in the FPU and you keep on  
>> reusing
>> them, you can keep an improved acuracy. But, as soon as you need to put  
>> your
>> results anywhere else, you fall back to 64 bits acuracy.
>
>   Not really. You can store the numbers in memory using the 80-bit
> accuracy. In C/C++ you can usually do that with the 'long double' type.

Not with a Microsoft C++ compiler though. In their infinite wisdom they  
chose the 64-bit type for both 'double' and 'long double'.


>   I don't know, however, if there's a speed penalty in storing/reading
> memory in units larger than 64 bits...

There was on the original Pentium. Newer processors may have wider data  
paths though.


-- 
FE

Post a reply to this message

Fredrik Eriksson <noo### [at] nowherecom> wrote:
> Not with a Microsoft C++ compiler though. In their infinite wisdom they  
> chose the 64-bit type for both 'double' and 'long double'.

  It's the same wisdom that makes 'long' 32-bit even when compiling to
a 64-bit binary... (This is the only compiler in the world that I know
of which does that.)

-- 
                                                          - Warp

Post a reply to this message

"Mike Sobers" <sob### [at] mindspringcom> wrote:

> Your question was "why do some people think it could be twice as fast?".
> It's because some people understand math and binary operations, but not
> neccessarily how computer architecture is designed around them.  That's why
> the question was asked in the first place, because a lot of us have a lot to
> learn about what advantages the new higher-precision hardware/software will
> provide.
>
> Mike

Well said! And, I might add, it shouldn't be necessary for us POV users to
have to become computer scientists or programming/architecture experts just
in order to understand why 64-bit POV on a 64-bit machine is faster (or
slower) than the 32-bit one. Simple, real-world benchmarks would
suffice...not a lot of holier-than-though technical browbeating.

Ken W.

Post a reply to this message

Kenneth <kdw### [at] earthlinknet> wrote:
> And, I might add, it shouldn't be necessary for us POV users to
> have to become computer scientists or programming/architecture experts just
> in order to understand why 64-bit POV on a 64-bit machine is faster (or
> slower) than the 32-bit one.

  If someone doesn't understand the difference between a 32-bit and a
64-bit processor and has the misconception that the latter will be
twice (or at least considerably) faster than the former and makes a
buying decision based on that misconception, he is up to a disappointment.
  I don't think it's a bad thing to know the real difference.

-- 
                                                          - Warp

Post a reply to this message

Warp <war### [at] tagpovrayorg> wrote:
> Kenneth <kdw### [at] earthlinknet> wrote:
> > And, I might add, it shouldn't be necessary for us POV users to
> > have to become computer scientists or programming/architecture experts just
> > in order to understand why 64-bit POV on a 64-bit machine is faster (or
> > slower) than the 32-bit one.
>
>   If someone doesn't understand the difference between a 32-bit and a
> 64-bit processor and has the misconception that the latter will be
> twice (or at least considerably) faster than the former and makes a
> buying decision based on that misconception, he is up to a disappointment.
>   I don't think it's a bad thing to know the real difference.
>
> --
>                                                           - Warp

Yes, but how much arcane knowledge are any of us, as individuals, expected
to know? Echoing the sentiments expressed earlier, we all have to pick and
choose what we devote our time to learning.  My refrigerator keeps my food
cold, but do I really need to know how the refrigeration cycle works in
order to use it (or buy one?) It's a bit unfair to assume that all of us
out here *should* know all about the inner workings of 64-bit processors,
and to bludgeon us if we don't. Processor architecture may be one of your
own prime interests, I don't know; but there are many of us (judging from
these posts) that *just don't need to know* such stuff. (Hey, I'm still
trying to learn POV-Ray!)

HOWEVER, that said, I do fall into the situation you've described, having
bought a 64-bit machine--an emachines T6412-- in the (seemingly mistaken!)
belief that it WOULD run POV-Ray "twice as fast" as as equivalent 32-bit
machine.  A belief based on all of the non-technical ideas expressed here
so far-- basically that 64-bit sounds like it is "twice" 32-bit. A dumb
belief? No. I think it comes from a lot of different ingredients (not the
least of which is computer manufacturer's hype.) It is indeed unfortunate
that computers, more so than just about any other technical gadget, seem to
require so VAST a store of knowledge in order to to understand (and to
purchase wisely.) Impossibly vast, IMHO. But am I disppointed with my
purchase?  Not at all! The only machine I can compare it to is my old
32-bit Pentium II 400MHZ. And it runs (32-bit) POV-Ray 6X faster. That's
all I really need to know!

SO...will it run 64-bit POV-Ray faster still? I haven't yet installed that
version, but I'm inclined to think not, judging from all that has been
discussed so far. That's disappointing.

Ken W.

Post a reply to this message

My favorite is "Long Boolean" where true="0000000000000001" and
false="0000000000000000"



"Warp" <war### [at] tagpovrayorg> wrote in message
news:44d92e04@news.povray.org...
> Fredrik Eriksson <noo### [at] nowherecom> wrote:
> > Not with a Microsoft C++ compiler though. In their infinite wisdom they
> > chose the 64-bit type for both 'double' and 'long double'.
>
>   It's the same wisdom that makes 'long' 32-bit even when compiling to
> a 64-bit binary... (This is the only compiler in the world that I know
> of which does that.)
>
> -- 
>                                                           - Warp

Post a reply to this message

Warp  wrote in message <44cdf9bd$1@news.povray.org>:
>    Now, the AMD64 is a special case: 64-bit binaries optimized for the
> AMD64 run slightly faster than 32-bit binaries optimized for the same
> processor. However, this speedup has nothing to do with the amount of
> bits used. It is because in 64-bit mode the CPU has enhancements which
> the binary can use for faster performance (the biggest enhancement is
> a bunch of additional CPU registers).

There is another point. I just remarked that the following code:

    for(i = 0; i <= N; i++)
	r += (double)i * (i / 100000000);

runs four to six times faster in 64 bits mode than in 32 bits mode on AMD64
or its intel clone, with gcc. This probably has to do with gcc using
different FPU mechanisms (SSE2 vs. x87). I am currently waiting for some of
my friends to test with different compilers and/or explain what they
understand of SSE2.

Post a reply to this message

Nicolas George <nicolas$george@salle-s.org> wrote:
>     for(i = 0; i <= N; i++)
>         r += (double)i * (i / 100000000);

  What is the type of 'i' there?

-- 
                                                          - Warp

Post a reply to this message

Warp  wrote in message <44f1fe00@news.povray.org>:
> Nicolas George <nicolas$george@salle-s.org> wrote:
>>     for(i = 0; i <= N; i++)
>>         r += (double)i * (i / 100000000);
> 
>   What is the type of 'i' there?

An unsigned; but it should not change anything, since it is the very same
code in both modes. Well, it could have been a long long, but I am not that
careless; and in fact, the difference between 64-bit integer arithmetic in
both modes is much smaller. Some of the missing parts are:

#define N 1000000000

static double
calc_double(void)
{
    unsigned i;
    register double r = 0; /* or not register */

    for(i = 0; i <= N; i++)
        r += (double)i * (i / 100000000);
    return(r);
}

    getrusage(RUSAGE_SELF, &t0);
    r1 = calc_double();
    getrusage(RUSAGE_SELF, &t1);

(the rest is just necessary noise)

Post a reply to this message

Nicolas George <nicolas$george@salle-s.org> wrote:
> Warp  wrote in message <44f1fe00@news.povray.org>:
> > Nicolas George <nicolas$george@salle-s.org> wrote:
> >>     for(i = 0; i <= N; i++)
> >>         r += (double)i * (i / 100000000);
> > 
> >   What is the type of 'i' there?

> An unsigned; but it should not change anything

  Well, I asked because I suspected it was an integer, in which case
the above code doesn't make too much sense.

  In C (and C++) the sentence "(i / 100000000)" is 0 for any (unsigned)
value of i less than 100000000. That because both arguments being integer,
that's an integer division, not a floating point one.
  Then, of course, the first 'i' is multiplied by 0, so 'r' is repeatedly
added '0' to it, which makes no difference.

  I don't know if this makes a difference in speed in different processors,
but it *might* be that the AMD64 can smartly perform those operations
(which always yield '0' as the answer) faster with its 64-bit-mode
enhancements. But not necessarily. It's just a guess.

  Anyways, what you probably wanted to do is this:

        r += (double)i * (i / 100000000.0);

which makes much more sense. Could you test if it that also is considerably
faster in 64-bit mode?

>     register double r = 0; /* or not register */

  The 'register' keyword is a no-op. It doesn't do anything and is just
a useless backwards-compatible drag of history.
  (Besides, even when it meant something (like 20 years ago or so), you
couldn't possibly use it with a 'double' because registers are used for
integers only.)

-- 
                                                          - Warp

Post a reply to this message

Warp  wrote in message <44f2271e@news.povray.org>:
>   I don't know if this makes a difference in speed in different processors,
> but it *might* be that the AMD64 can smartly perform those operations
> (which always yield '0' as the answer) faster with its 64-bit-mode
> enhancements. But not necessarily. It's just a guess.

No, it does not chance anything at all.

I made that benchmark in the first time to compare floating point arithmetic
and 64 bits integer arithmetic, to select which one to use for a situation
where both would be acceptable. At first, I just used "r += i * i", but I
added the division to get a result that fits in 64 bits, and to be able to
compare it to a trusted result. Adding it barely changed the timings at all.

>   The 'register' keyword is a no-op. It doesn't do anything and is just
> a useless backwards-compatible drag of history.

I tried it because I suspected the 80 bits / 64 bits problem: maybe gcc
would feel freer to keep the double in 80 bits precision. But it did not
change anything.

Post a reply to this message

Nicolas George <nicolas$george@salle-s.org> wrote:
> I tried it because I suspected the 80 bits / 64 bits problem: maybe gcc
> would feel freer to keep the double in 80 bits precision.

  'double' is 64-bit in x86 architectures (and in fact in most other
architectures too). If you want a 80-bit fp number, use 'long double'.

> But it did not change anything.

  As I said, the 'register' keyword is a no-op. I would bet that most
if not all modern compilers simply ignore it completely (except when
checking for correct syntactical usage, just for backwards compatibility;
after that check they most probably just drop it completely).

-- 
                                                          - Warp

Post a reply to this message

Nicolas George wrote:
> I made that benchmark in the first time to compare floating point arithmetic
> and 64 bits integer arithmetic, to select which one to use for a situation
> where both would be acceptable. At first, I just used "r += i * i", but I
> added the division to get a result that fits in 64 bits, and to be able to
> compare it to a trusted result. Adding it barely changed the timings at all.

You do realise that "unsigned" declares an "unsigned int", which will be 32
bit in both 32 and 64 bit modes of compilation? Thus, the only thing your
code as posted tests is how well the compiler handles the slowest operations
in there, which are the 32 bit integer division and the conversion of a 32
bit integer to a double. Neither of which tell you much about performance of
anything but those two specific operations, which are special cases in and
by themselves.

In fact, the integer division is slow due to the iterative algorithm (in
hardware!) needed to execute it. It would in fact be slower for most 64 bit
integers assuming the numbers you divide are sufficiently big (outside 32
bit range) because the time an integer division takes is not constant but
rather depends on the number of one- and zero-bits with any reasonable
processor released in the last three decades.

	Thorsten

Post a reply to this message

Warp wrote:
>   As I said, the 'register' keyword is a no-op. I would bet that most
> if not all modern compilers simply ignore it completely (except when
> checking for correct syntactical usage, just for backwards compatibility;
> after that check they most probably just drop it completely).

Please be advised that your statement is incorrect. 'register' is not
defined to be a "no-op" at all. See for example ISO/IEC 14882:2003 (or 1998
if you like, either way it is the ISO C++ standard) section 7.1.1 number 3.
Specifically, it is a hint to the compiler that "the object so declared will
be heavily used". Of course, it is true that some modern compilers
completely ignore it, but that is more due to their register allocators not
being able to handle external priority hints rather than the keyword being
specified to be a "no-op" anywhere. Also note that is is completely legal to
declare a float or double as 'register', it I completely reasonable to
expect a compiler to be able to use this hint to optimise the usage of
floating-point registers (except that a x87 FPU does not have registers but
a stack, most RISC processors do of course have a full complement of
floating-point registers).

	Thorsten

Post a reply to this message

Thorsten Froehlich <tho### [at] trfde> wrote:
> Please be advised that your statement is incorrect. 'register' is not
> defined to be a "no-op" at all. See for example ISO/IEC 14882:2003 (or 1998
> if you like, either way it is the ISO C++ standard) section 7.1.1 number 3.
> Specifically, it is a hint to the compiler that "the object so declared will
> be heavily used".

  I meant "it's in practice a no-op". I didn't mean it's "officially a
deprecated keyword".

  Of course in theory it gives a hint to the compiler, and in really old
C compilers it really had an effect, but AFAIK compilers have ignored it
completely for over a decade. They will use their own internal optimization
algorithms regardless of that keyword.

  It's the same as with the 'inline' keyword with regard to optimization:
When optimizing, compilers usually completely ignore that keyword. It doesn't
have any effect on the output. (But of course 'inline' has other meaningful
uses which makes it a non-no-op.)

-- 
                                                          - Warp

Post a reply to this message

Thorsten Froehlich <tho### [at] trfde> wrote:
> In fact, the integer division is slow due to the iterative algorithm (in
> hardware!) needed to execute it. It would in fact be slower for most 64 bit
> integers assuming the numbers you divide are sufficiently big (outside 32
> bit range) because the time an integer division takes is not constant but
> rather depends on the number of one- and zero-bits with any reasonable
> processor released in the last three decades.

  Are you sure it's an iterative algorithm at transistor level?

  One would think that, for example, adding two integers is iterative
(how else could you know if the sum of two bits overflows, thus creating
the need to add an additional 1 to the next bit?). However, it is
perfectly possible to add integers of any amount of bits in 1 clock cycle.

-- 
                                                          - Warp

Post a reply to this message

Warp wrote:
> Thorsten Froehlich <tho### [at] trfde> wrote:
>> In fact, the integer division is slow due to the iterative algorithm (in
>> hardware!) needed to execute it. It would in fact be slower for most 64 bit
>> integers assuming the numbers you divide are sufficiently big (outside 32
>> bit range) because the time an integer division takes is not constant but
>> rather depends on the number of one- and zero-bits with any reasonable
>> processor released in the last three decades.
> 
>   Are you sure it's an iterative algorithm at transistor level?

Yes. I don't have a good reference at hand, but a very quick search reveals
<http://www.intel.com/technology/itj/2006/volume10issue02/art01_Intro_to_Core_Duo/p03_improved_cores.htm>.
Just search for "IDIV" in that document and notice the remark about the
algorithm being iterative and them employing "early exit", which accounts
for the variable execution time (aka number of zeros and ones in the input).
Given that this talks about Intel's most modern CPU core, implications
should be clear.

Of course, all this is much better explained in books about computer
hardware (plenty of those around). The good ones present several variations
of algorithms available in detail. However, in essence the algorithms remain
rather similar to a binary version of the common decimal "by hand" division
method we all once learned in school. As you may recall, that "algorithm" is
also iterative ;-)

	Thorsten

Post a reply to this message

Thorsten Froehlich  wrote in message <44f2e083$1@news.povray.org>:
> You do realise that "unsigned" declares an "unsigned int", which will be 32
> bit in both 32 and 64 bit modes of compilation? Thus, the only thing your
> code as posted tests is how well the compiler handles the slowest operations
> in there, which are the 32 bit integer division and the conversion of a 32
> bit integer to a double. Neither of which tell you much about performance of
> anything but those two specific operations, which are special cases in and
> by themselves.

I made a benchmark suited to the specific case I wanted to optimize¹, and it
served me. I stumbled on the difference for floats as a side effect, and
just reported it in this thread because it seemed relevant.

After digging in the assembly code, I found the exact cause, which is a
combination of architecture improvements and bad compiler optimisation. As
you guessed, the bottleneck is the integer to float conversion (the
division, on the other hand, is meaningless, since it is the very same in 32
and 64 bits).

In 64 bits, gcc uses cvtsi2sdq to convert the integer into a SSE2 register.
In 32 bits, the default is to use the FPU, and the conversion goes through
the stack; in 32 bits with SSE2, the conversion goes through the stack,
through the FPU and again through the stack, which is a (known) very bad
optimisation by gcc. Converting to use cvtsi2sd gives results similar to the
64 bits version.

As a conclusion, it is possible to sum up with:

- (Already explained here) 64 bits processors are also a new generation of
  processors, with new features and a better architecture, that lead to
  better performances both in 64 bits and 32 bits mode.

- (New remark) 64 bits processors need a rewrite of the compiler's code
  generation phase, that can provide improved usage of the new features.


1 : if you want to know everything, my goal is to compute mean square error
and linear correlation coefficient of video image, where the most repeated
operation is something like "t += *d * *d". Since there is a very large
number of pixels, t needs to be either a 64 bits integer or a double
precision float. I found that 64 bits integer were slightly faster on 64
bits CPU, and not too much slower on 32 bits CPU.

Post a reply to this message