POV-Ray : Newsgroups : povray.beta-test : Cached Macros! Server Time
29 Mar 2024 05:37:34 EDT (-0400)
  Cached Macros! (Message 11 to 20 of 32)  
<<< Previous 10 Messages Goto Latest 10 Messages Next 10 Messages >>>
From: clipka
Subject: Re: Cached Macros!
Date: 18 Jul 2016 07:55:27
Message: <578cc3af$1@news.povray.org>
Am 18.07.2016 um 13:29 schrieb Jaime Vives Piqueres:
> El 18/07/16 a las 11:48, clipka escribió:
>> I'd be interested to hear from the Linux jockeys how they are faring
>> with in-file macro invocation. I'd expect a similar picture as with
>> Windows, but the "sweet spot" may be around 0.5k worth of code
>> between macro declaration and invocation, since that's the size of
>> the buffer on Unix. Then again, Unix may provide additional buffers
>> for file access -- there must be a reason why POV-Ray's own buffers
>> where chosen that small on the Linux platform.
> 
>   Using Thomas example on my recently installed Ubuntu 16.04:
> 
>   current alpha, inside : 1m20s
>   current alpha, outside: 1m20s
>   previous alpha, inside: 8m11s
>   previous alpha, outside: 12m46
> 
>   Can I just say "wow!"?

I presume the "inside" is still with the random macros residing in
"rand.inc", right?
Remember that under those conditions, that's still mostly a test of the
"outside" macros.

>> One thing seriously worrying me, however, is that the version using
>> cached macros exhibits seriously worse parsing performance on the
>> 2nd and any subsequent runs, taking about 100 seconds instead of 60
>> seconds. Not sure yet what's going on there.
> 
>   Cannot see that here... subsequent runs still take 1m20s.

Note that "subsequent runs" on Windows means the executable isn't
terminated between runs. On Linux, the closest equivalent would be
subsequent frames in an animation.


Post a reply to this message

From: Jaime Vives Piqueres
Subject: Re: Cached Macros!
Date: 18 Jul 2016 08:46:28
Message: <578ccfa4$1@news.povray.org>
El 18/07/16 a las 13:55, clipka escribió:
> I presume the "inside" is still with the random macros residing in
> "rand.inc", right?

   Yes...

> Remember that under those conditions, that's still mostly a test of
> the "outside" macros.

   Ok, will try a more "pure inside" test later.

> Note that "subsequent runs" on Windows means the executable isn't
> terminated between runs. On Linux, the closest equivalent would be
> subsequent frames in an animation.

   Ah... then I just traced a few frames: the first took 1m20s, and
subsequent ones took 1m24s. Not a big increase.

--
jaime


Post a reply to this message

From: clipka
Subject: Re: Cached Macros!
Date: 18 Jul 2016 08:55:40
Message: <578cd1cc$1@news.povray.org>
Am 18.07.2016 um 13:49 schrieb Chris Cason:
> On 15/07/2016 09:15, clipka wrote:
>> I have now implemented a mechanism that keeps a copy of each and every
>> macro's code in memory (except for macros more than 65536 characters in
>> size), greatly increasing the execution speed of macros in such
>> scenarios, to the point that said recommendation should be obsolete from
>> now on.
> 
> Nice improvement there Christoph :)

Dunno. I'm just realizing that I've implemented in the wrong place: By
using the existing IMemStream, which parallels basic file access as
implemented by the IStream family, parsing of cached macros currently
still uses the file buffer mentioned in one of the other posts, being
implemented in the ITextStream. This means some unnecessary bulk data
copies in memory.

I'm now pondering whether to solve this by implementing an unbuffered
alternative to ITextStream, which would still access an underlying
IStream, or whether to implement an alternative to ITextStream that
reads directly from the macro cache.


Post a reply to this message

From: Thomas de Groot
Subject: Re: Cached Macros!
Date: 18 Jul 2016 08:59:07
Message: <578cd29b$1@news.povray.org>
On 18-7-2016 11:48, clipka wrote:
> Am 18.07.2016 um 10:05 schrieb Thomas de Groot:
>> On 15-7-2016 1:15, clipka wrote:
>>> While I'm satisfied with the cross-file macro execution speed, I would
>>> like additional feedback from you guys about execution speed of macros
>>> declared in the same file. I see a slight increase there, too -- can you
>>> confirm?
>>
>> Slight? You are kidding! :-)
>
> Actually no, I'm not.
>
>> The speed increase is huge in fact compared to version 3.7.
>> With the attached quick-and-dirty scene I get the following results
>> using 1 million iterations:
>>
>> +av151; macro inside scene parsing time: 1 minute 5 seconds (65.988
>> seconds)
>> +av151; macro outside scene parsing time: 1 minute 6 seconds (66.971
>> seconds)
>>
>> V 3.7; macro inside scene parsing time: stopped parsing manually after
>> more than 10 minutes!
>
> I was quite puzzled about your results, until I saw your test scene, and
> found that the in-file macro you presumably tested actually calls
> various other macros, which are declared in rand.inc. So what you are
> actually measuring is the speed of macro invocations across file boundaries.
>
> With the macros RRand, VRand_In_Box and VRand_In_Obj moved to the scene
> itself and "rand.inc" no longer included, I actually see a noteworthy
> _slowdown_ in parsing time on my machine: about 60 seconds, as opposed
> to 45 seconds with 3.7.0.

You are right of course. I am so used to this kind of scene building 
that I forgot the rand.inc implications. I shall have to retest this 
better tomorrow.


-- 
Thomas


Post a reply to this message

From: Jaime Vives Piqueres
Subject: Re: Cached Macros!
Date: 18 Jul 2016 13:49:17
Message: <578D169D.3000705@ignorancia.org>
El 18/07/16 a las 13:55, clipka escribió:
> Am 18.07.2016 um 13:29 schrieb Jaime Vives Piqueres:
>>    Using Thomas example on my recently installed Ubuntu 16.04:
>>
>>    current alpha, inside : 1m20s
>>    current alpha, outside: 1m20s
>>    previous alpha, inside: 8m11s
>>    previous alpha, outside: 12m46
>>
>>    Can I just say "wow!"?
>
> I presume the "inside" is still with the random macros residing in
> "rand.inc", right?

   Ok, revised results with "everything inside" (copy-paste of 3 macros
from rand.inc"):

   current alpha : 1m20s
   previous alpha: 1m20s

   kinda logical, isn't?


--
jaime


Post a reply to this message

From: clipka
Subject: Re: Cached Macros!
Date: 18 Jul 2016 17:05:10
Message: <578d4486$1@news.povray.org>
Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:

>   Ok, revised results with "everything inside" (copy-paste of 3 macros
> from rand.inc"):
> 
>   current alpha : 1m20s
>   previous alpha: 1m20s

What happens if you put >512 bytes between the macro declarations and
the place where they're invoked?

>   kinda logical, isn't?

No, not necessarily.

With the new version, macros are always executed from the cache (unless
they're larger than 64k), while previously they were executed from
buffered file access. The two approaches involve different types of
overhead, so I'd normally expect them to actually result in at least
slightly different execution times.


Post a reply to this message

From: Jaime Vives Piqueres
Subject: Re: Cached Macros!
Date: 18 Jul 2016 17:27:55
Message: <578d49db$1@news.povray.org>
El 18/07/16 a las 23:05, clipka escribió:
> Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:
>
>>    Ok, revised results with "everything inside" (copy-paste of 3 macros
>> from rand.inc"):
>>
>>    current alpha : 1m20s
>>    previous alpha: 1m20s
>
> What happens if you put >512 bytes between the macro declarations and
> the place where they're invoked?

   Nothing... still 1m20s.


>>    kinda logical, isn't?
>
> No, not necessarily.
>
> With the new version, macros are always executed from the cache (unless
> they're larger than 64k), while previously they were executed from
> buffered file access. The two approaches involve different types of
> overhead, so I'd normally expect them to actually result in at least
> slightly different execution times.
>

   The difference must be really small in this case then...

--
jaime


Post a reply to this message

From: clipka
Subject: Re: Cached Macros!
Date: 18 Jul 2016 17:47:33
Message: <578d4e75$1@news.povray.org>
Am 18.07.2016 um 23:27 schrieb Jaime Vives Piqueres:
> El 18/07/16 a las 23:05, clipka escribió:
>> Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:
>>
>>>    Ok, revised results with "everything inside" (copy-paste of 3 macros
>>> from rand.inc"):
>>>
>>>    current alpha : 1m20s
>>>    previous alpha: 1m20s
>>
>> What happens if you put >512 bytes between the macro declarations and
>> the place where they're invoked?
> 
>   Nothing... still 1m20s.

Did you verify that you're testing the right versions?

>   The difference must be really small in this case then...

Apparently.


Post a reply to this message

From: Thomas de Groot
Subject: Re: Cached Macros!
Date: 19 Jul 2016 03:28:34
Message: <578dd6a2$1@news.povray.org>
On 18-7-2016 11:48, clipka wrote:
> With the macros RRand, VRand_In_Box and VRand_In_Obj moved to the scene
> itself and "rand.inc" no longer included, I actually see a noteworthy
> _slowdown_ in parsing time on my machine: about 60 seconds, as opposed
> to 45 seconds with 3.7.0.

Correct. the 1 minute 5 seconds I mentioned previously, against 54 
seconds in standard version.

-- 
Thomas


Post a reply to this message

From: Jaime Vives Piqueres
Subject: Re: Cached Macros!
Date: 19 Jul 2016 07:45:26
Message: <578e12d6@news.povray.org>
El 18/07/16 a las 23:47, clipka escribió:
> Am 18.07.2016 um 23:27 schrieb Jaime Vives Piqueres:
>> El 18/07/16 a las 23:05, clipka escribió:
>>> Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:
>>>
>>>>    Ok, revised results with "everything inside" (copy-paste of 3 macros
>>>> from rand.inc"):
>>>>
>>>>    current alpha : 1m20s
>>>>    previous alpha: 1m20s
>>>
>>> What happens if you put >512 bytes between the macro declarations and
>>> the place where they're invoked?
>>
>>   Nothing... still 1m20s.
>
> Did you verify that you're testing the right versions?

  Yes, I just checked the test was using correct versions. Now, testing a
bit further, I don't get any noticeable increase in parse time until I
increase the extra in-between comments to 2017 bytes. With 2016 bytes it
still parses on 1m20s, but with 2017 it jumps to 1m24s and keeps stable
at that time even if I increase the extra comments further.


--
jaime


Post a reply to this message

<<< Previous 10 Messages Goto Latest 10 Messages Next 10 Messages >>>

Copyright 2003-2023 Persistence of Vision Raytracer Pty. Ltd.