Am 18.07.2016 um 13:49 schrieb Chris Cason:
> On 15/07/2016 09:15, clipka wrote:
>> I have now implemented a mechanism that keeps a copy of each and every
>> macro's code in memory (except for macros more than 65536 characters in
>> size), greatly increasing the execution speed of macros in such
>> scenarios, to the point that said recommendation should be obsolete from
>> now on.
> 
> Nice improvement there Christoph :)

Dunno. I'm just realizing that I've implemented in the wrong place: By
using the existing IMemStream, which parallels basic file access as
implemented by the IStream family, parsing of cached macros currently
still uses the file buffer mentioned in one of the other posts, being
implemented in the ITextStream. This means some unnecessary bulk data
copies in memory.

I'm now pondering whether to solve this by implementing an unbuffered
alternative to ITextStream, which would still access an underlying
IStream, or whether to implement an alternative to ITextStream that
reads directly from the macro cache.

Post a reply to this message

On 18-7-2016 11:48, clipka wrote:
> Am 18.07.2016 um 10:05 schrieb Thomas de Groot:
>> On 15-7-2016 1:15, clipka wrote:
>>> While I'm satisfied with the cross-file macro execution speed, I would
>>> like additional feedback from you guys about execution speed of macros
>>> declared in the same file. I see a slight increase there, too -- can you
>>> confirm?
>>
>> Slight? You are kidding! :-)
>
> Actually no, I'm not.
>
>> The speed increase is huge in fact compared to version 3.7.
>> With the attached quick-and-dirty scene I get the following results
>> using 1 million iterations:
>>
>> +av151; macro inside scene parsing time: 1 minute 5 seconds (65.988
>> seconds)
>> +av151; macro outside scene parsing time: 1 minute 6 seconds (66.971
>> seconds)
>>
>> V 3.7; macro inside scene parsing time: stopped parsing manually after
>> more than 10 minutes!
>
> I was quite puzzled about your results, until I saw your test scene, and
> found that the in-file macro you presumably tested actually calls
> various other macros, which are declared in rand.inc. So what you are
> actually measuring is the speed of macro invocations across file boundaries.
>
> With the macros RRand, VRand_In_Box and VRand_In_Obj moved to the scene
> itself and "rand.inc" no longer included, I actually see a noteworthy
> _slowdown_ in parsing time on my machine: about 60 seconds, as opposed
> to 45 seconds with 3.7.0.

You are right of course. I am so used to this kind of scene building 
that I forgot the rand.inc implications. I shall have to retest this 
better tomorrow.


-- 
Thomas

Post a reply to this message

El 18/07/16 a las 13:55, clipka escribió:
> Am 18.07.2016 um 13:29 schrieb Jaime Vives Piqueres:
>>    Using Thomas example on my recently installed Ubuntu 16.04:
>>
>>    current alpha, inside : 1m20s
>>    current alpha, outside: 1m20s
>>    previous alpha, inside: 8m11s
>>    previous alpha, outside: 12m46
>>
>>    Can I just say "wow!"?
>
> I presume the "inside" is still with the random macros residing in
> "rand.inc", right?

   Ok, revised results with "everything inside" (copy-paste of 3 macros
from rand.inc"):

   current alpha : 1m20s
   previous alpha: 1m20s

   kinda logical, isn't?


--
jaime

Post a reply to this message

Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:

>   Ok, revised results with "everything inside" (copy-paste of 3 macros
> from rand.inc"):
> 
>   current alpha : 1m20s
>   previous alpha: 1m20s

What happens if you put >512 bytes between the macro declarations and
the place where they're invoked?

>   kinda logical, isn't?

No, not necessarily.

With the new version, macros are always executed from the cache (unless
they're larger than 64k), while previously they were executed from
buffered file access. The two approaches involve different types of
overhead, so I'd normally expect them to actually result in at least
slightly different execution times.

Post a reply to this message

El 18/07/16 a las 23:05, clipka escribió:
> Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:
>
>>    Ok, revised results with "everything inside" (copy-paste of 3 macros
>> from rand.inc"):
>>
>>    current alpha : 1m20s
>>    previous alpha: 1m20s
>
> What happens if you put >512 bytes between the macro declarations and
> the place where they're invoked?

   Nothing... still 1m20s.


>>    kinda logical, isn't?
>
> No, not necessarily.
>
> With the new version, macros are always executed from the cache (unless
> they're larger than 64k), while previously they were executed from
> buffered file access. The two approaches involve different types of
> overhead, so I'd normally expect them to actually result in at least
> slightly different execution times.
>

   The difference must be really small in this case then...

--
jaime

Post a reply to this message

Am 18.07.2016 um 23:27 schrieb Jaime Vives Piqueres:
> El 18/07/16 a las 23:05, clipka escribió:
>> Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:
>>
>>>    Ok, revised results with "everything inside" (copy-paste of 3 macros
>>> from rand.inc"):
>>>
>>>    current alpha : 1m20s
>>>    previous alpha: 1m20s
>>
>> What happens if you put >512 bytes between the macro declarations and
>> the place where they're invoked?
> 
>   Nothing... still 1m20s.

Did you verify that you're testing the right versions?

>   The difference must be really small in this case then...

Apparently.

Post a reply to this message

On 18-7-2016 11:48, clipka wrote:
> With the macros RRand, VRand_In_Box and VRand_In_Obj moved to the scene
> itself and "rand.inc" no longer included, I actually see a noteworthy
> _slowdown_ in parsing time on my machine: about 60 seconds, as opposed
> to 45 seconds with 3.7.0.

Correct. the 1 minute 5 seconds I mentioned previously, against 54 
seconds in standard version.

-- 
Thomas

Post a reply to this message

El 18/07/16 a las 23:47, clipka escribió:
> Am 18.07.2016 um 23:27 schrieb Jaime Vives Piqueres:
>> El 18/07/16 a las 23:05, clipka escribió:
>>> Am 18.07.2016 um 19:49 schrieb Jaime Vives Piqueres:
>>>
>>>>    Ok, revised results with "everything inside" (copy-paste of 3 macros
>>>> from rand.inc"):
>>>>
>>>>    current alpha : 1m20s
>>>>    previous alpha: 1m20s
>>>
>>> What happens if you put >512 bytes between the macro declarations and
>>> the place where they're invoked?
>>
>>   Nothing... still 1m20s.
>
> Did you verify that you're testing the right versions?

  Yes, I just checked the test was using correct versions. Now, testing a
bit further, I don't get any noticeable increase in parse time until I
increase the extra in-between comments to 2017 bytes. With 2016 bytes it
still parses on 1m20s, but with 2017 it jumps to 1m24s and keeps stable
at that time even if I increase the extra comments further.


--
jaime

Post a reply to this message

Ok, folks -- I've just submitted an update to the macro caching
mechanism. The 64k (Windows) / 0.5k (Linux) buffer is now bypassed,
further increasing the parsing speed of Thomas' demo scene on my machine
from 60 to 50 seconds.

Please test intensively.

Post a reply to this message

On 20-7-2016 1:16, clipka wrote:
> Ok, folks -- I've just submitted an update to the macro caching
> mechanism. The 64k (Windows) / 0.5k (Linux) buffer is now bypassed,
> further increasing the parsing speed of Thomas' demo scene on my machine
> from 60 to 50 seconds.
>
> Please test intensively.
>

Looking good. I tested with/without little counter:

av155; macros in; counter yes: 57 sec (57.283)
av155; macros in; counter no: 53 sec (53.836)
av155; macros out; counter yes: 1 min 10 sec (70.232)
av155; macros out; counter no: 1 min 6 sec (66.862)

v3.7.0; macros in; counter yes: 56 sec (56.145)
v3.7.0; macros in; counter no: 53 sec (53.118)

I leave the in-between comments testing to Jaime ;-)

-- 
Thomas

Post a reply to this message