Hi Folks,

Until now, each and every invocation of a macro from a file other than
the one it was declared in, inevitably caused the latter file to be
opened again for the macro code to be re-read -- a process that
seriously bogged down the parser. It therefore used to be recommended to
copy frequently used macros into the main scene file (or whatever file
they were called from).

I have now implemented a mechanism that keeps a copy of each and every
macro's code in memory (except for macros more than 65536 characters in
size), greatly increasing the execution speed of macros in such
scenarios, to the point that said recommendation should be obsolete from
now on.


While I'm satisfied with the cross-file macro execution speed, I would
like additional feedback from you guys about execution speed of macros
declared in the same file. I see a slight increase there, too -- can you
confirm?

Also, I'd like you folks to go wild trying to find scenarios where the
caching of macros has any drawbacks, and of course find that one bug I
have hidden there.


As usual, both Windows binaries and source code packages can be found at
https://github.com/POV-Ray/povray/releases. The source code can also be
found as the current master branch at https://github.com/POV-Ray/povray.

(The source code in question carries the version number
v3.7.1-alpha.8697421, the Windows binaries carry the build suffix +av151.)

Post a reply to this message

On 7/14/2016 7:15 PM, clipka wrote:
> I have now implemented a mechanism

when i went to document this ... noticed syntax diagram /might/ be 
missing the term optional in a couple of places. agreed?

http://wiki.povray.org/content/Reference:User_Defined_Macros

Post a reply to this message

On 15-7-2016 1:15, clipka wrote:
> While I'm satisfied with the cross-file macro execution speed, I would
> like additional feedback from you guys about execution speed of macros
> declared in the same file. I see a slight increase there, too -- can you
> confirm?

Slight? You are kidding! :-)
The speed increase is huge in fact compared to version 3.7.
With the attached quick-and-dirty scene I get the following results 
using 1 million iterations:

+av151; macro inside scene parsing time: 1 minute 5 seconds (65.988 seconds)
+av151; macro outside scene parsing time: 1 minute 6 seconds (66.971 
seconds)

V 3.7; macro inside scene parsing time: stopped parsing manually after 
more than 10 minutes!

Note: for using the test scene with outside macro, just copy the macro 
to another place and uncomment the macro's include.

>
> Also, I'd like you folks to go wild trying to find scenarios where the
> caching of macros has any drawbacks, and of course find that one bug I
> have hidden there.
>

Not found any yet.

-- 
Thomas

Post a reply to this message

Attachments:
Download 'clipka_cached macros_test.pov.txt' (2 KB)

Am 17.07.2016 um 21:25 schrieb Jim Holsenback:
> On 7/14/2016 7:15 PM, clipka wrote:
>> I have now implemented a mechanism
> 
> when i went to document this ... noticed syntax diagram /might/ be
> missing the term optional in a couple of places. agreed?
> 
> http://wiki.povray.org/content/Reference:User_Defined_Macros

I guess that's a matter of philosophy: Do you want the documentation to
generally reflect the newest version (even if that's still under
development), with side notes mentioning what features aren't available
in earlier versions, or do you want the documentation to generally
reflect a well-established version, with side notes mentioning the
features added in newer version?

So far I have been following the latter approach for documenting the new
3.7.1 features, mostly because 3.7.1 hasn't even reached beta phase yet;
theoretically, any modifications since 3.7.0 are still subject to
change, and 3.7.0 is still the latest version for which we provide a
Windows installer.

But if you are significantly more comfortable with the former approach,
I can live with that as well.

Post a reply to this message

Am 18.07.2016 um 10:05 schrieb Thomas de Groot:
> On 15-7-2016 1:15, clipka wrote:
>> While I'm satisfied with the cross-file macro execution speed, I would
>> like additional feedback from you guys about execution speed of macros
>> declared in the same file. I see a slight increase there, too -- can you
>> confirm?
> 
> Slight? You are kidding! :-)

Actually no, I'm not.

> The speed increase is huge in fact compared to version 3.7.
> With the attached quick-and-dirty scene I get the following results
> using 1 million iterations:
> 
> +av151; macro inside scene parsing time: 1 minute 5 seconds (65.988
> seconds)
> +av151; macro outside scene parsing time: 1 minute 6 seconds (66.971
> seconds)
> 
> V 3.7; macro inside scene parsing time: stopped parsing manually after
> more than 10 minutes!

I was quite puzzled about your results, until I saw your test scene, and
found that the in-file macro you presumably tested actually calls
various other macros, which are declared in rand.inc. So what you are
actually measuring is the speed of macro invocations across file boundaries.

With the macros RRand, VRand_In_Box and VRand_In_Obj moved to the scene
itself and "rand.inc" no longer included, I actually see a noteworthy
_slowdown_ in parsing time on my machine: about 60 seconds, as opposed
to 45 seconds with 3.7.0.


The picture changes a bit when the scene file is large -- or, more
precisely, when the macro is declared far away from where it is invoked:
With approximately 64k worth of comments inserted between the macro
declaration and its invocation, the parse time increases to about 95
seconds with 3.7.0, while remaining at about 60 seconds with the new
version.

This is to be expected, since the parser uses buffered file access, with
a 64k buffer on Windows machines, so in 3.7.0 a macro invoked from near
its declaration may still be included in the buffer, in which case
parsing can proceed without actual file access, while a farther-away
macro will trigger a 64k read to fill the buffer with different content.
It's still a good deal faster than cross-file invocation, since there is
no need to re-open another file. As for why macro caching is slower than
3.7.0's "near invocation", that's because it also adds a bit of overhead
every time a macro is invoked from the cache.

I'd be interested to hear from the Linux jockeys how they are faring
with in-file macro invocation. I'd expect a similar picture as with
Windows, but the "sweet spot" may be around 0.5k worth of code between
macro declaration and invocation, since that's the size of the buffer on
Unix. Then again, Unix may provide additional buffers for file access --
there must be a reason why POV-Ray's own buffers where chosen that small
on the Linux platform.


One thing seriously worrying me, however, is that the version using
cached macros exhibits seriously worse parsing performance on the 2nd
and any subsequent runs, taking about 100 seconds instead of 60 seconds.
Not sure yet what's going on there.

Post a reply to this message

On 7/18/2016 4:17 AM, clipka wrote:
> Am 17.07.2016 um 21:25 schrieb Jim Holsenback:
>> On 7/14/2016 7:15 PM, clipka wrote:
>>> I have now implemented a mechanism
>>
>> when i went to document this ... noticed syntax diagram /might/ be
>> missing the term optional in a couple of places. agreed?
>>
>> http://wiki.povray.org/content/Reference:User_Defined_Macros
>
> I guess that's a matter of philosophy: Do you want the documentation to
> generally reflect the newest version (even if that's still under
> development), with side notes mentioning what features aren't available
> in earlier versions, or do you want the documentation to generally
> reflect a well-established version, with side notes mentioning the
> features added in newer version?
>
> So far I have been following the latter approach for documenting the new
> 3.7.1 features, mostly because 3.7.1 hasn't even reached beta phase yet;
> theoretically, any modifications since 3.7.0 are still subject to
> change, and 3.7.0 is still the latest version for which we provide a
> Windows installer.
>
> But if you are significantly more comfortable with the former approach,
> I can live with that as well.
>

well yikes ... was hoping for a less vague response. i was just trying 
to be proactive. back when the new / change template request was 
suggested i roughed something out quickly but didn't get a chance to 
implement, then noticed it's usage in the documentation. couple of day 
ago had some time to put towards this and finished up the templates, 
found ALL the old usage and changed to new, even added code to 
wikidocgen to translate template to it's html equivalent. now i'm unsure 
what to do next because i was under the impression that we were running 
as fast as we can towards 3.7.1 release ... well like i said now i'm not 
sure anymore

Post a reply to this message

On 7/18/2016 6:08 AM, Jim Holsenback wrote:
> On 7/18/2016 4:17 AM, clipka wrote:
>> Am 17.07.2016 um 21:25 schrieb Jim Holsenback:
>>> On 7/14/2016 7:15 PM, clipka wrote:
>>>> I have now implemented a mechanism
>>>
>>> when i went to document this ... noticed syntax diagram /might/ be
>>> missing the term optional in a couple of places. agreed?
>>>
>>> http://wiki.povray.org/content/Reference:User_Defined_Macros
>>
>> I guess that's a matter of philosophy: Do you want the documentation to
>> generally reflect the newest version (even if that's still under
>> development), with side notes mentioning what features aren't available
>> in earlier versions, or do you want the documentation to generally
>> reflect a well-established version, with side notes mentioning the
>> features added in newer version?
>>
>> So far I have been following the latter approach for documenting the new
>> 3.7.1 features, mostly because 3.7.1 hasn't even reached beta phase yet;
>> theoretically, any modifications since 3.7.0 are still subject to
>> change, and 3.7.0 is still the latest version for which we provide a
>> Windows installer.
>>
>> But if you are significantly more comfortable with the former approach,
>> I can live with that as well.
>>
>
> well yikes ... was hoping for a less vague response. i was just trying
> to be proactive. back when the new / change template request was
> suggested i roughed something out quickly but didn't get a chance to
> implement, then noticed it's usage in the documentation. couple of day
> ago had some time to put towards this and finished up the templates,
> found ALL the old usage and changed to new, even added code to
> wikidocgen to translate template to it's html equivalent. now i'm unsure
> what to do next because i was under the impression that we were running
> as fast as we can towards 3.7.1 release ... well like i said now i'm not
> sure anymore

just a follow up ... to address the concern that i /might/ be getting 
ahead of the game here.

Welcome to the POV-Ray version 3.7.x documentation repository ... if 
you're looking for the most up to date documentation, you've found it! 
This content is used to generate the documentation included with the 
distribution, so occasionally it can get ahead of the current release.

this preamble /is/ there for /everyone/ to see at:
http://wiki.povray.org/content/Documentation:Contents

Post a reply to this message

El 18/07/16 a las 11:48, clipka escribió:
> I'd be interested to hear from the Linux jockeys how they are faring
> with in-file macro invocation. I'd expect a similar picture as with
> Windows, but the "sweet spot" may be around 0.5k worth of code
> between macro declaration and invocation, since that's the size of
> the buffer on Unix. Then again, Unix may provide additional buffers
> for file access -- there must be a reason why POV-Ray's own buffers
> where chosen that small on the Linux platform.

   Using Thomas example on my recently installed Ubuntu 16.04:

   current alpha, inside : 1m20s
   current alpha, outside: 1m20s
   previous alpha, inside: 8m11s
   previous alpha, outside: 12m46

   Can I just say "wow!"?


> One thing seriously worrying me, however, is that the version using
> cached macros exhibits seriously worse parsing performance on the
> 2nd and any subsequent runs, taking about 100 seconds instead of 60
> seconds. Not sure yet what's going on there.
>

   Cannot see that here... subsequent runs still take 1m20s.


--
jaime

Post a reply to this message

On 18/07/2016 20:30, Jim Holsenback wrote:
> http://wiki.povray.org/content/Reference:User_Defined_Macros

[snip]

> this preamble /is/ there for /everyone/ to see at:
> http://wiki.povray.org/content/Documentation:Contents

I like how you've done these. Seems pretty clear to me.

-- Chris

Post a reply to this message

On 15/07/2016 09:15, clipka wrote:
> I have now implemented a mechanism that keeps a copy of each and every
> macro's code in memory (except for macros more than 65536 characters in
> size), greatly increasing the execution speed of macros in such
> scenarios, to the point that said recommendation should be obsolete from
> now on.

Nice improvement there Christoph :)

-- Chris

Post a reply to this message

Am 18.07.2016 um 13:29 schrieb Jaime Vives Piqueres:
> El 18/07/16 a las 11:48, clipka escribió:
>> I'd be interested to hear from the Linux jockeys how they are faring
>> with in-file macro invocation. I'd expect a similar picture as with
>> Windows, but the "sweet spot" may be around 0.5k worth of code
>> between macro declaration and invocation, since that's the size of
>> the buffer on Unix. Then again, Unix may provide additional buffers
>> for file access -- there must be a reason why POV-Ray's own buffers
>> where chosen that small on the Linux platform.
> 
>   Using Thomas example on my recently installed Ubuntu 16.04:
> 
>   current alpha, inside : 1m20s
>   current alpha, outside: 1m20s
>   previous alpha, inside: 8m11s
>   previous alpha, outside: 12m46
> 
>   Can I just say "wow!"?

I presume the "inside" is still with the random macros residing in
"rand.inc", right?
Remember that under those conditions, that's still mostly a test of the
"outside" macros.

>> One thing seriously worrying me, however, is that the version using
>> cached macros exhibits seriously worse parsing performance on the
>> 2nd and any subsequent runs, taking about 100 seconds instead of 60
>> seconds. Not sure yet what's going on there.
> 
>   Cannot see that here... subsequent runs still take 1m20s.

Note that "subsequent runs" on Windows means the executable isn't
terminated between runs. On Linux, the closest equivalent would be
subsequent frames in an animation.

Post a reply to this message

El 18/07/16 a las 13:55, clipka escribió:
> I presume the "inside" is still with the random macros residing in
> "rand.inc", right?

   Yes...

> Remember that under those conditions, that's still mostly a test of
> the "outside" macros.

   Ok, will try a more "pure inside" test later.

> Note that "subsequent runs" on Windows means the executable isn't
> terminated between runs. On Linux, the closest equivalent would be
> subsequent frames in an animation.

   Ah... then I just traced a few frames: the first took 1m20s, and
subsequent ones took 1m24s. Not a big increase.

--
jaime

Post a reply to this message

Am 18.07.2016 um 13:49 schrieb Chris Cason:
> On 15/07/2016 09:15, clipka wrote:
>> I have now implemented a mechanism that keeps a copy of each and every
>> macro's code in memory (except for macros more than 65536 characters in
>> size), greatly increasing the execution speed of macros in such
>> scenarios, to the point that said recommendation should be obsolete from
>> now on.
> 
> Nice improvement there Christoph :)

Dunno. I'm just realizing that I've implemented in the wrong place: By
using the existing IMemStream, which parallels basic file access as
implemented by the IStream family, parsing of cached macros currently
still uses the file buffer mentioned in one of the other posts, being
implemented in the ITextStream. This means some unnecessary bulk data
copies in memory.

I'm now pondering whether to solve this by implementing an unbuffered
alternative to ITextStream, which would still access an underlying
IStream, or whether to implement an alternative to ITextStream that
reads directly from the macro cache.

Post a reply to this message

On 18-7-2016 11:48, clipka wrote:
> Am 18.07.2016 um 10:05 schrieb Thomas de Groot:
>> On 15-7-2016 1:15, clipka wrote:
>>> While I'm satisfied with the cross-file macro execution speed, I would
>>> like additional feedback from you guys about execution speed of macros
>>> declared in the same file. I see a slight increase there, too -- can you
>>> confirm?
>>
>> Slight? You are kidding! :-)
>
> Actually no, I'm not.
>
>> The speed increase is huge in fact compared to version 3.7.
>> With the attached quick-and-dirty scene I get the following results
>> using 1 million iterations:
>>
>> +av151; macro inside scene parsing time: 1 minute 5 seconds (65.988
>> seconds)
>> +av151; macro outside scene parsing time: 1 minute 6 seconds (66.971
>> seconds)
>>
>> V 3.7; macro inside scene parsing time: stopped parsing manually after
>> more than 10 minutes!
>
> I was quite puzzled about your results, until I saw your test scene, and
> found that the in-file macro you presumably tested actually calls
> various other macros, which are declared in rand.inc. So what you are
> actually measuring is the speed of macro invocations across file boundaries.
>
> With the macros RRand, VRand_In_Box and VRand_In_Obj moved to the scene
> itself and "rand.inc" no longer included, I actually see a noteworthy
> _slowdown_ in parsing time on my machine: about 60 seconds, as opposed
> to 45 seconds with 3.7.0.

You are right of course. I am so used to this kind of scene building 
that I forgot the rand.inc implications. I shall have to retest this 
better tomorrow.


-- 
Thomas

Post a reply to this message

El 18/07/16 a las 13:55, clipka escribió:
> Am 18.07.2016 um 13:29 schrieb Jaime Vives Piqueres:
>>    Using Thomas example on my recently installed Ubuntu 16.04:
>>
>>    current alpha, inside : 1m20s
>>    current alpha, outside: 1m20s
>>    previous alpha, inside: 8m11s
>>    previous alpha, outside: 12m46
>>
>>    Can I just say "wow!"?
>
> I presume the "inside" is still with the random macros residing in
> "rand.inc", right?

   Ok, revised results with "everything inside" (copy-paste of 3 macros
from rand.inc"):

   current alpha : 1m20s
   previous alpha: 1m20s

   kinda logical, isn't?


--
jaime

Post a reply to this message

Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:

>   Ok, revised results with "everything inside" (copy-paste of 3 macros
> from rand.inc"):
> 
>   current alpha : 1m20s
>   previous alpha: 1m20s

What happens if you put >512 bytes between the macro declarations and
the place where they're invoked?

>   kinda logical, isn't?

No, not necessarily.

With the new version, macros are always executed from the cache (unless
they're larger than 64k), while previously they were executed from
buffered file access. The two approaches involve different types of
overhead, so I'd normally expect them to actually result in at least
slightly different execution times.

Post a reply to this message

El 18/07/16 a las 23:05, clipka escribió:
> Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:
>
>>    Ok, revised results with "everything inside" (copy-paste of 3 macros
>> from rand.inc"):
>>
>>    current alpha : 1m20s
>>    previous alpha: 1m20s
>
> What happens if you put >512 bytes between the macro declarations and
> the place where they're invoked?

   Nothing... still 1m20s.


>>    kinda logical, isn't?
>
> No, not necessarily.
>
> With the new version, macros are always executed from the cache (unless
> they're larger than 64k), while previously they were executed from
> buffered file access. The two approaches involve different types of
> overhead, so I'd normally expect them to actually result in at least
> slightly different execution times.
>

   The difference must be really small in this case then...

--
jaime

Post a reply to this message

Am 18.07.2016 um 23:27 schrieb Jaime Vives Piqueres:
> El 18/07/16 a las 23:05, clipka escribió:
>> Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:
>>
>>>    Ok, revised results with "everything inside" (copy-paste of 3 macros
>>> from rand.inc"):
>>>
>>>    current alpha : 1m20s
>>>    previous alpha: 1m20s
>>
>> What happens if you put >512 bytes between the macro declarations and
>> the place where they're invoked?
> 
>   Nothing... still 1m20s.

Did you verify that you're testing the right versions?

>   The difference must be really small in this case then...

Apparently.

Post a reply to this message

On 18-7-2016 11:48, clipka wrote:
> With the macros RRand, VRand_In_Box and VRand_In_Obj moved to the scene
> itself and "rand.inc" no longer included, I actually see a noteworthy
> _slowdown_ in parsing time on my machine: about 60 seconds, as opposed
> to 45 seconds with 3.7.0.

Correct. the 1 minute 5 seconds I mentioned previously, against 54 
seconds in standard version.

-- 
Thomas

Post a reply to this message

El 18/07/16 a las 23:47, clipka escribió:
> Am 18.07.2016 um 23:27 schrieb Jaime Vives Piqueres:
>> El 18/07/16 a las 23:05, clipka escribió:
>>> Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:
>>>
>>>>    Ok, revised results with "everything inside" (copy-paste of 3 macros
>>>> from rand.inc"):
>>>>
>>>>    current alpha : 1m20s
>>>>    previous alpha: 1m20s
>>>
>>> What happens if you put >512 bytes between the macro declarations and
>>> the place where they're invoked?
>>
>>   Nothing... still 1m20s.
>
> Did you verify that you're testing the right versions?

  Yes, I just checked the test was using correct versions. Now, testing a
bit further, I don't get any noticeable increase in parse time until I
increase the extra in-between comments to 2017 bytes. With 2016 bytes it
still parses on 1m20s, but with 2017 it jumps to 1m24s and keeps stable
at that time even if I increase the extra comments further.


--
jaime

Post a reply to this message

Ok, folks -- I've just submitted an update to the macro caching
mechanism. The 64k (Windows) / 0.5k (Linux) buffer is now bypassed,
further increasing the parsing speed of Thomas' demo scene on my machine
from 60 to 50 seconds.

Please test intensively.

Post a reply to this message

On 20-7-2016 1:16, clipka wrote:
> Ok, folks -- I've just submitted an update to the macro caching
> mechanism. The 64k (Windows) / 0.5k (Linux) buffer is now bypassed,
> further increasing the parsing speed of Thomas' demo scene on my machine
> from 60 to 50 seconds.
>
> Please test intensively.
>

Looking good. I tested with/without little counter:

av155; macros in; counter yes: 57 sec (57.283)
av155; macros in; counter no: 53 sec (53.836)
av155; macros out; counter yes: 1 min 10 sec (70.232)
av155; macros out; counter no: 1 min 6 sec (66.862)

v3.7.0; macros in; counter yes: 56 sec (56.145)
v3.7.0; macros in; counter no: 53 sec (53.118)

I leave the in-between comments testing to Jaime ;-)

-- 
Thomas

Post a reply to this message

El 20/07/16 a las 01:16, clipka escribió:
> Ok, folks -- I've just submitted an update to the macro caching
> mechanism. The 64k (Windows) / 0.5k (Linux) buffer is now bypassed,
> further increasing the parsing speed of Thomas' demo scene on my
> machine from 60 to 50 seconds.
>
> Please test intensively.
>

   Hmmm... here the new version is a bit slower parsing: from 1m20s to
1m23s, either with macros defined inside or outside, and even with
in-between comments no matter the size (I tried several MB of comments
and it still parsed in 1m23s).


3.7.1-alpha.8704732.unofficial (g++ 5.4.0 @ x86_64-pc-linux-gnu)

--
jaime

Post a reply to this message

Am 20.07.2016 um 12:40 schrieb Jaime Vives Piqueres:
> El 20/07/16 a las 01:16, clipka escribió:
>> Ok, folks -- I've just submitted an update to the macro caching
>> mechanism. The 64k (Windows) / 0.5k (Linux) buffer is now bypassed,
>> further increasing the parsing speed of Thomas' demo scene on my
>> machine from 60 to 50 seconds.
>>
>> Please test intensively.
>>
> 
>   Hmmm... here the new version is a bit slower parsing: from 1m20s to
> 1m23s, either with macros defined inside or outside, and even with
> in-between comments no matter the size (I tried several MB of comments
> and it still parsed in 1m23s).

Hmm... maybe that's due to the top layer in the file access
(ITextStream, the one where the buffer used to reside in) now using
polymorphism. Still, it puzzles me a bit why the copying-around of the
macro content does so little to counteract that penalty. Might even have
to do with CPU properties.

Since this 3.6% performance loss on your machine buys a 12% performance
bonus on Thomas' machine, I guess I'm willing to pay that price.

Post a reply to this message

On 7/20/2016 12:43 PM, clipka wrote:
> Since this 3.6% performance loss on your machine buys a 12% performance
> bonus on Thomas' machine, I guess I'm willing to pay that price.

You put that in, to be commented upon. Didn't you? ;)

-- 

Regards
     Stephen

Post a reply to this message

On 07/19/2016 07:16 PM, clipka wrote:
> Ok, folks -- I've just submitted an update to the macro caching
> mechanism. The 64k (Windows) / 0.5k (Linux) buffer is now bypassed,
> further increasing the parsing speed of Thomas' demo scene on my machine
> from 60 to 50 seconds.
>
> Please test intensively.
>

Ubuntu 14.04 (i3-4130). Latest always came in faster for me using a 
scene of my own. Inserted an error statement to stop after parsing so 
these times are for the parsing phase. Command suffix the commit.

-------------------------- Disk ----------------------------
povray371_2965368:
Command exited with non-zero status 1
158.46user 3.64system 2:42.42elapsed 99%CPU (0avgtext+0avgdata 
151536maxresident)k
0inputs+6608outputs (0major+28992minor)pagefaults 0swap

povray371_8e965a9:                       -8.60% (elapsed)
Command exited with non-zero status 1
148.02user 0.08system 2:28.45elapsed 99%CPU (0avgtext+0avgdata 
151548maxresident)k
0inputs+6608outputs (0major+28482minor)pagefaults 0swaps

-------------------------- Memory disk ---------------------
povray371_2965368:
Command exited with non-zero status 1
153.68user 3.26system 2:37.26elapsed 99%CPU (0avgtext+0avgdata 
151676maxresident)k
0inputs+0outputs (0major+28482minor)pagefaults 0swaps

povray371_8e965a9:
Command exited with non-zero status 1    -3.85% (elapsed)
150.72user 0.14system 2:31.20elapsed 99%CPU (0avgtext+0avgdata 
151460maxresident)k
0inputs+0outputs (0major+28477minor)pagefaults 0swap

-------------------------- Memory disk (one file) ----------
povray371_2965368:
Command exited with non-zero status 1
151.63user 1.63system 2:33.59elapsed 99%CPU (0avgtext+0avgdata 
151628maxresident)k
0inputs+0outputs (0major+28479minor)pagefaults 0swaps

povray371_8e965a9:
Command exited with non-zero status 1    -3.37% (elapsed)
148.01user 0.07system 2:28.41elapsed 99%CPU (0avgtext+0avgdata 
151556maxresident)k
0inputs+0outputs (0major+28988minor)pagefaults 0swaps

Post a reply to this message

Am 20.07.2016 um 14:15 schrieb Stephen:
> On 7/20/2016 12:43 PM, clipka wrote:
>> Since this 3.6% performance loss on your machine buys a 12% performance
>> bonus on Thomas' machine, I guess I'm willing to pay that price.
> 
> You put that in, to be commented upon. Didn't you? ;)

Absolutely. Substantial ones preferred, thanks :P

Post a reply to this message

Am 20.07.2016 um 16:59 schrieb William F Pokorny:

> povray371_2965368:

Just wondering -- which one is that? I failed to find any commit with
that id in the repository.

Post a reply to this message

On 07/20/2016 11:23 AM, clipka wrote:
> Am 20.07.2016 um 16:59 schrieb William F Pokorny:
>
>> povray371_2965368:
>
> Just wondering -- which one is that? I failed to find any commit with
> that id in the repository.
>

POV-Ray 3.7.1-alpha.8697045.unofficial

No idea what happened...

git log --oneline currently shows me:

8e965a9 Improved cached macro performance.
fa3a1dc Fixed a code flaw in macro caching.
39a6c9f [ci skip] Minor changes to comments and dev documentation.
edfae26 Fixed a bug in cleanup of fog with turbulence.
3b5b4eb added missing file pavement_s3_t6_p12.txt / issue #37 (#77)
051b674 Implemented macro caching.
2d0a42c fix filenames / issue #36 (#76)

And looking back through my bash command history 2d0a42c is what I 
checked out ahead of the compile to the pre-cached macro version.

"git checkout 2d0a42c"

Bill P.

Post a reply to this message

El 20/07/16 a las 13:43, clipka escribió:
> Hmm... maybe that's due to the top layer in the file access
> (ITextStream, the one where the buffer used to reside in) now using
> polymorphism. Still, it puzzles me a bit why the copying-around of
> the macro content does so little to counteract that penalty. Might
> even have to do with CPU properties.

   AMD FX-6300 here...

> Since this 3.6% performance loss on your machine buys a 12%
> performance bonus on Thomas' machine, I guess I'm willing to pay that
> price.
>

   Well, it's still a little loss compared to the huge gain respect to
the old macros mechanism. Good job!

--
jaime

Post a reply to this message

On 20-7-2016 14:15, Stephen wrote:
> On 7/20/2016 12:43 PM, clipka wrote:
>> Since this 3.6% performance loss on your machine buys a 12% performance
>> bonus on Thomas' machine, I guess I'm willing to pay that price.
>
> You put that in, to be commented upon. Didn't you? ;)
>

I thought so too. So, I agree with Jaime that the principal gain is on 
the outside use of macros and that is more than a bonus. Excellent job 
indeed!

-- 
Thomas

Post a reply to this message

On 2016-07-19 23:16:48 +0000, clipka said:

> Ok, folks -- I've just submitted an update to the macro caching
> mechanism. The 64k (Windows) / 0.5k (Linux) buffer is now bypassed,
> further increasing the parsing speed of Thomas' demo scene on my machine
> from 60 to 50 seconds.
> 
> Please test intensively.

And on Macintosh:

- 3.7.0    
allInside: 32,957 sec 
allOutside: 96,369 sec  
rodOutsideRandInside:  481,025 sec  !! (8minutes)

- av155
allInside: 37,021 sec
allOutside:  36,480 sec
rodOutsideRandInside: 36,869 sec



Files used:

// Persistence of Vision Ray Tracer Scene Description File
// File: Clipka_cached macros_test.pov
// Vers: 3.7.1 Cached Macros version
// Desc: Basic Scene Example
// Date: mm/dd/yy
// Auth: ?
//

#version 3.7;

#include "colors.inc"
#declare allInside = 1;
#declare allOutside = 2;
#declare rodOutsideRandInside = 3;
#declare MacroType = allInside;

#if ( (MacroType = allInside) | (MacroType = rodOutsideRandInside))
	 #debug "\nRand inside"
	#macro VRand_In_Box(Mn, Mx, RS) (< rand(RS), rand(RS), 
rand(RS)>*(Mx-Mn) + Mn) #end
	#macro RRand(Min, Max, RS) (rand(RS)*(Max-Min) + Min) #end
	#macro VRand_In_Obj(Obj, RS)
    		#local Mn = min_extent(Obj);
   		 #local Mx = max_extent(Obj);
    		#local Pt = VRand_In_Box(Mn, Mx, RS);
    		#local J = 0;
   		 #while(inside(Obj, Pt) = 0 & J < 1000)
        		#local Pt = VRand_In_Box(Mn, Mx, RS);
        		#local J = J + 1;
    		#end
    	(Pt)
	#end
#end //if (MacroType = allInside || MacroType = rodOutsideRandInside)

#if (MacroType = allInside )
	 #debug "\nRod inside"
	#macro Rods()
	#ifndef(Box)	
		#declare Box =
		box {
  			<-10, 0, -10>,<10, 0.2, 10>
  			translate 10*y
		}
		#declare Rand = seed(1101);
	#end //#ifndef(Box)

	#declare Norm  = <0, 0, 0>;
	#declare Start = VRand_In_Obj(Box, Rand);
	#declare Pos   = trace (Surface, Start, -y, Norm);   

	#declare Cyl =
	cylinder {
 		 0, <0,1,0>, 0.05
 		 scale <1, RRand(0.5, 1.5, Rand), 1>
  		translate Pos
  		pigment {srgb <RRand(0.1, 0.9, Rand), RRand(0.1, 0.9, Rand), 
RRand(0.1, 0.9, Rand)>}
	}
	#end //	#macro Rods()
#end //#if (MacroType = allInside || MacroType = allOutside)

#if (MacroType = allOutside   | MacroType = rodOutsideRandInside)
	#include "rods.inc"
#end

global_settings {
  assumed_gamma 1.0
}

// ----------------------------------------

camera {
  location  <0.0, 5, -40.0>
  direction 1.5*z
  right     x*image_width/image_height
  look_at   <0.0, 0.0,  0.0>
}

sky_sphere {
  pigment {
    gradient y
    color_map {
      [0.0 srgb <0.6,0.7,1.0>*1.3]
      [0.7 srgb <0.0,0.1,0.8>*1.3]
    }
  }
}

light_source {
  <0, 0, 0>            // light's position (translated below)
  color rgb <1, 1, 1>  // light's color
  translate <-3, 3, -3>*1000
}

// ----------------------------------------

#declare Surface =
plane {
  y, 0
  pigment { color srgb <0.7, 0.5, 0.3> }
}

object {Surface}



#declare I=0;

#for (I, 0, 1000000)
Rods()
object {Cyl}

#if (mod(I,10000) = 0)    
  #debug concat("\nRods: ", str(I,4,0)) 
#end 
#end



----------------------------------------------------------------------------------------------------------------------


//File: rods.inc

#if (MacroType = allOutside)
	 #debug "\nRand outside"

	#macro VRand_In_Box(Mn, Mx, RS) (< rand(RS), rand(RS), 
rand(RS)>*(Mx-Mn) + Mn) #end
	#macro RRand(Min, Max, RS) (rand(RS)*(Max-Min) + Min) #end
	#macro VRand_In_Obj(Obj, RS)
	    #local Mn = min_extent(Obj);
	    #local Mx = max_extent(Obj);
	    #local Pt = VRand_In_Box(Mn, Mx, RS);
	    #local J = 0;
	    #while(inside(Obj, Pt) = 0 & J < 1000)
	        #local Pt = VRand_In_Box(Mn, Mx, RS);
	        #local J = J + 1;
	    #end
	    (Pt)
	#end //#macro VRand_In_Obj(Obj, RS)
#end //#if (MacroType = allOutside)

#if (MacroType = allOutside | (MacroType = rodOutsideRandInside))
	 #debug "\nRod outside"
	#macro Rods()
		#ifndef(Box)
		#declare Box =
		box {
		  <-10, 0, -10>,<10, 0.2, 10>
		  translate 10*y
		}
		#declare Rand = seed(1101);
		#end
		
		#declare Norm  = <0, 0, 0>;
		#declare Start = VRand_In_Obj(Box, Rand);
		#declare Pos   = trace (Surface, Start, -y, Norm);   
		
		#declare Cyl =
		cylinder {
		  0, <0,1,0>, 0.05
		  scale <1, RRand(0.5, 1.5, Rand), 1>
		  translate Pos
		  pigment {srgb <RRand(0.1, 0.9, Rand), RRand(0.1, 0.9, Rand), 
RRand(0.1, 0.9, Rand)>}
		}
	#end	
#end
-- 
-------------------------------------------------------------------------------------------


POV-Ray 3.7 unofficial: http://megapov.inetart.net/povrayunofficial_mac/
UberPOV Mac: http://megapov.inetart.net/uberpov_mac/index.html#Mac
MegaPOV: http://megapov.inetart.net
E-mail: yvo(DOT)s(AT)gmx.net

Post a reply to this message

Attachments:
Download 'iso-8859-1' (17 KB)