POV-Ray : Newsgroups : povray.beta-test : Restructured Parser wants Testing Server Time
29 Nov 2022 05:37:14 EST (-0500)
  Restructured Parser wants Testing (Message 11 to 20 of 59)  
<<< Previous 10 Messages Goto Latest 10 Messages Next 10 Messages >>>
From: Stephen
Subject: Re: Restructured Parser wants Testing
Date: 23 May 2018 04:22:18
Message: <5b0524ba$1@news.povray.org>
On 22/05/2018 01:28, clipka wrote:
> - Backward compatibility with scenes that use single backslashes in
> literal filenames has been sacrificed, and will most likely not be restored.

This is going to cause me problems using SDL generated by modellers.

Test:

v 3.8.0-alpha.9475849+av541.msvc14 is faster than
v 3.8.0-x. tokenizer.9999+av5 60.msvc

On the second run of the file. The parse time was a second or two slower 
than the first time. Except when using Ver 3.7 when it was the same.

Statistic Files attached



-- 

Regards
     Stephen


Post a reply to this message


Attachments:
Download 'hot_v 3.8.0-alpha.9475849+av541.msvc14.txt' (3 KB) Download 'hot_v 3.8.0-x. tokenizer.9999+av5 60.msvc.txt' (3 KB) Download 'hot_version 3.7.0.msvc10.win64_.txt' (3 KB) Download 'v 3.8.0-alpha.9475849+av541.msvc14.txt' (3 KB) Download 'v 3.8.0-x. tokenizer.9999+av5 60.msvc.txt' (3 KB) Download 'version 3.7.0.msvc10.win64_.txt' (3 KB)

From: clipka
Subject: Re: Restructured Parser wants Testing
Date: 23 May 2018 08:20:01
Message: <5b055c71$1@news.povray.org>
Am 23.05.2018 um 08:54 schrieb Thomas de Groot:

> Where the longer parse time comes from is in the while loop distributing
> the cubes. This loop resides in a separate inc file. and looks like this:
> 
> //==========================================================
> #local R1 = seed(1929);
> #local R2 = seed(9732);
> #local R3 = seed(6493);
> #local N1 = 4000;
> #local I  = 0;
> 
> #while (I <= N1)
>   #local Start = VRand_In_Obj(RocksBox, R1);
                   ^^^^^^^^^^^^
Unless you copied that to your file with the loop, that's a macro in a
different file...

>   #local Norm = <0, 0, 0>;
>   #local RockPos = trace(Landscape, Start, -y, Norm);
>   #declare Visible = IsObjectVisible(RockPos)
>   #if (Visible)
>     #if (vdot(Norm, y)>0.8 & RockPos.y<50.0*(0.8+rand(R1)*0.4) &
> RockF(RockPos.x,RockPos.y,RockPos.z)<0.5+rand(R1)*0.1 & RockPos.y>0 )
>       object {Rock
>         //scale (RockPos.y+(1/RockPos.y))*RRand(0.9, 1.1, R2)
                                            ^^^^^
>         scale RockPos.y*RRand(0.9, 1.1, R2)
                          ^^^^^
>         rotate RRand(-180, 180, R3)
                 ^^^^^
... as is this.

>         translate RockPos
>       }
>       #if (mod(I, N1/10) = 0)
>         #debug concat(" Rocks: ", str(I,5,0), "\n")
>       #end
>       #local I = I + 1;
>     #end  //slope & altitude
>   #end  //Visible
> #end

So yes, that pretty much explains the poor performance with the
overhauled parser -- or, more to the point, the better performance with
v3.8.0-alpha: The latter features cached macros, while the overhauled
parser currently has those disabled.

I would expect v3.7.0 to show roughly similar performance as the
overhauled parser.


Post a reply to this message

From: clipka
Subject: Re: Restructured Parser wants Testing
Date: 23 May 2018 08:26:22
Message: <5b055dee$1@news.povray.org>
Am 23.05.2018 um 10:22 schrieb Stephen:
> On 22/05/2018 01:28, clipka wrote:
>> - Backward compatibility with scenes that use single backslashes in
>> literal filenames has been sacrificed, and will most likely not be
>> restored.
> 
> This is going to cause me problems using SDL generated by modellers.

Care to do some name calling?

Techically, those modellers would have to be considered broken: To the
best of my knowledge, single backslashes as filename separators have
never been an officially supported feature.

That said, if the modeller in question should happen to be of reasonable
significance despite no longer being actively maintained, I may go back
to the drawing board after all.


Post a reply to this message

From: clipka
Subject: Re: Restructured Parser wants Testing
Date: 23 May 2018 08:55:05
Message: <5b0564a9$1@news.povray.org>
Am 23.05.2018 um 10:22 schrieb Stephen:

> Test:
> 
> v 3.8.0-alpha.9475849+av541.msvc14 is faster than
> v 3.8.0-x. tokenizer.9999+av5 60.msvc
> 
> On the second run of the file. The parse time was a second or two slower
> than the first time. Except when using Ver 3.7 when it was the same.

I take that as good news, presuming the scene in question does call
macros across files. Such a scenario would explain the superior
performance of v3.8.0-alpha over v3.7.0 as the result of macro caching,
as well as the inferior performance of v3.8.0-x.tokenizer vs.
v3.8.0-alpha as the result of disabled macro caching.

What really excites me is the superior performance of v3.8.0-x.tokenizer
over v3.7.0. It means that with macro caching re-enabled it should not
only break even with, but actually surpass, v3.8.0-alpha performance.

Mind you, I haven't even started with the actual speed improvements that
I am aiming for. All I've been doing so far was adding structure (and
thus potential overhead) to the parser while trying to minimize added
overhead. Apparently I've actually reduced some existing overhead along
the way.


Post a reply to this message

From: clipka
Subject: Re: Restructured Parser wants Testing
Date: 23 May 2018 09:18:04
Message: <5b056a0c@news.povray.org>
Am 22.05.2018 um 02:28 schrieb clipka:
> Please give this version a thorough shakedown, with focus on parsing:
> 
> https://github.com/POV-Ray/povray/releases/tag/v3.8.0-x.tokenizer.9999
> 
> Besides testing functionality, please also look at parsing speed.

Thanks to all who have tested for parsing speed so far. I'm quite
confident now that v3.8.0-x.tokenizer is on the right track in that
respect, so my main worry now is whether the SDL functionality is all
there (except of course the portions I have deliberately disabled for
now), without any bugs having crept in.

The fact that none of you has reported any bugs yet makes me a bit
uneasy... the new code can't be /that/ good ;)

(Also, I intend to create a collection of test cases for future
regression testing of the parser, and was hoping for lots of bug reports
for inspiration. Looks like that plan is pretty much foiled ;))


Post a reply to this message

From: Stephen
Subject: Re: Restructured Parser wants Testing
Date: 23 May 2018 10:19:05
Message: <5b057859$1@news.povray.org>
On 23/05/2018 13:26, clipka wrote:
> Am 23.05.2018 um 10:22 schrieb Stephen:
>> On 22/05/2018 01:28, clipka wrote:
>>> - Backward compatibility with scenes that use single backslashes in
>>> literal filenames has been sacrificed, and will most likely not be
>>> restored.
>>
>> This is going to cause me problems using SDL generated by modellers.
> 
> Care to do some name calling?
> 

Pal / Best friend. ;-)

> Technically, those modellers would have to be considered broken: 

That seems fair enough.

> To the
> best of my knowledge, single backslashes as filename separators have
> never been an officially supported feature.
> 
By PovRay for Windows and/or for DOS?


> That said, if the modeller in question should happen to be of reasonable
> significance despite no longer being actively maintained, I may go back
> to the drawing board after all.
> 

Significance is relevant. I have no idea how many people use B3D. And I 
can't remember if Blender outputs the directory separator as a "\" or 
"/". (I have stopped using it until some of the bug reports I have 
submitted, have been fixed.)

It is possible to get B3D to export bespoke code. So we could set a flag 
telling the parser that the directory separator was "\".
e.g. windoze_separator = true
If that would help cut down the complexity.

Which reminds me. I use that facility to overwrite the version number 
B3D exports by declaring it after the first one. Will that workaround 
still work?

Best friend. :-)




-- 

Regards
     Stephen


Post a reply to this message

From: Stephen
Subject: Re: Restructured Parser wants Testing
Date: 23 May 2018 10:38:56
Message: <5b057d00@news.povray.org>
On 23/05/2018 13:55, clipka wrote:
> Am 23.05.2018 um 10:22 schrieb Stephen:
> 
>> Test:
>>
>> v 3.8.0-alpha.9475849+av541.msvc14 is faster than
>> v 3.8.0-x. tokenizer.9999+av5 60.msvc
>>
>> On the second run of the file. The parse time was a second or two slower
>> than the first time. Except when using Ver 3.7 when it was the same.
> 
> I take that as good news, presuming the scene in question does call
> macros across files. Such a scenario would explain the superior
> performance of v3.8.0-alpha over v3.7.0 as the result of macro caching,
> as well as the inferior performance of v3.8.0-x.tokenizer vs.
> v3.8.0-alpha as the result of disabled macro caching.
> 

Sorry it does not use macros. It did but I de-constructed the grass 
macro it was using as it was declaring an array of meshes from within a 
while loop. [Watch the memory usage rise and the HDD thrash.]


I have already had jr complain that I sent him the code in one file and 
not split into include files.
So, what is wrong with 54317 lines in one file, anyway? ;-)


> What really excites me is the superior performance of v3.8.0-x.tokenizer
> over v3.7.0. It means that with macro caching re-enabled it should not
> only break even with, but actually surpass, v3.8.0-alpha performance.
> 
> Mind you, I haven't even started with the actual speed improvements that
> I am aiming for. All I've been doing so far was adding structure (and
> thus potential overhead) to the parser while trying to minimize added
> overhead. Apparently I've actually reduced some existing overhead along
> the way.
> 


-- 

Regards
     Stephen


Post a reply to this message

From: clipka
Subject: Re: Restructured Parser wants Testing
Date: 23 May 2018 11:26:58
Message: <5b058842@news.povray.org>
Am 23.05.2018 um 16:18 schrieb Stephen:

>>> This is going to cause me problems using SDL generated by modellers.
>>
>> Care to do some name calling?
>>
> 
> Pal / Best friend. ;-)

Not exactly what I meant - I was rather trying to coax you into posting
the name of the modeller, thereby calling it broken ;)


>> Technically, those modellers would have to be considered broken: 
> 
> That seems fair enough.
> 
>> To the
>> best of my knowledge, single backslashes as filename separators have
>> never been an officially supported feature.
>>
> By PovRay for Windows and/or for DOS?

Technically, that statement was true for all of them: I know precious
little about POV-Ray for DOS, so to the best of my knowledge... ;)


But I have to revert that statement now, having dug up an older
documentation:

In POV-Ray 3.0, according to those docs, backslashes in string literals
was only supposed to take on a special role when followed by a
double-quote, in which case the backslash would be discarded while the
double-quote would be interpreted as a character rather than the end of
the string. Interpretation of other escape sequences was considered not
a feature of string literals, but rather of user message stream output
(`#debug` and its kin).


> Significance is relevant. I have no idea how many people use B3D. And I
> can't remember if Blender outputs the directory separator as a "\" or
> "/". (I have stopped using it until some of the bug reports I have
> submitted, have been fixed.)

Yeah, I guess B3D qualifies as relevant indeed.


> It is possible to get B3D to export bespoke code. So we could set a flag
> telling the parser that the directory separator was "\".
> e.g. windoze_separator = true
> If that would help cut down the complexity.

Not sure I understand what you mean. Would you change B3D, or would you
change POV-Ray?

When it comes to ideal format in POV-Ray scenes, it is highly
recommended to always use "/" as path separator. Works on all platforms,
'Doze incldued.


> Which reminds me. I use that facility to overwrite the version number
> B3D exports by declaring it after the first one. Will that workaround
> still work?

Uh... please elaborate. How would the resulting scene file look like?


Post a reply to this message

From: clipka
Subject: Re: Restructured Parser wants Testing
Date: 23 May 2018 11:30:49
Message: <5b058929$1@news.povray.org>
Am 23.05.2018 um 16:38 schrieb Stephen:
> On 23/05/2018 13:55, clipka wrote:
>> Am 23.05.2018 um 10:22 schrieb Stephen:
>>
>>> Test:
>>>
>>> v 3.8.0-alpha.9475849+av541.msvc14 is faster than
>>> v 3.8.0-x. tokenizer.9999+av5 60.msvc
>>>
>>> On the second run of the file. The parse time was a second or two slower
>>> than the first time. Except when using Ver 3.7 when it was the same.
>>
>> I take that as good news, presuming the scene in question does call
>> macros across files. Such a scenario would explain the superior
>> performance of v3.8.0-alpha over v3.7.0 as the result of macro caching,
>> as well as the inferior performance of v3.8.0-x.tokenizer vs.
>> v3.8.0-alpha as the result of disabled macro caching.
>>
> 
> Sorry it does not use macros. It did but I de-constructed the grass
> macro it was using as it was declaring an array of meshes from within a
> while loop. [Watch the memory usage rise and the HDD thrash.]

Hmm... then that begs the question, which change between v3.7.0 and
v3.8.0-alpha would have improved parsing speed?

Besides, what loop size are we talking about? Would that happen to be
>64k characters in the loop body?

> I have already had jr complain that I sent him the code in one file and
> not split into include files.
> So, what is wrong with 54317 lines in one file, anyway? ;-)

Nothing. It might actually be perfect for my parser tests. ;)


Post a reply to this message

From: Stephen
Subject: Re: Restructured Parser wants Testing
Date: 23 May 2018 12:53:40
Message: <5b059c94@news.povray.org>
On 23/05/2018 16:26, clipka wrote:
> Am 23.05.2018 um 16:18 schrieb Stephen:
> 
>>>> This is going to cause me problems using SDL generated by modellers.
>>>
>>> Care to do some name calling?
>>>
>>
>> Pal / Best friend. ;-)
> 
> Not exactly what I meant - I was rather trying to coax you into posting
> the name of the modeller, thereby calling it broken ;)
> 

Name and shame, then. No problem to me but you worked it out. Bishop3D

BTW As I keep saying to my wife, I don't do hints. You have to spell it 
out. Although not like her, in Morse code with a rubber mallet. ;)

3 dot, 2 dot 4 dot dash. ;)


> 
>>> Technically, those modellers would have to be considered broken:
>>
>> That seems fair enough.
>>
>>> To the
>>> best of my knowledge, single backslashes as filename separators have
>>> never been an officially supported feature.
>>>
>> By PovRay for Windows and/or for DOS?
> 
> Technically, that statement was true for all of them: I know precious
> little about POV-Ray for DOS, so to the best of my knowledge... ;)
> 
> 
> But I have to revert that statement now, having dug up an older
> documentation:
> 
> In POV-Ray 3.0, according to those docs, backslashes in string literals
> was only supposed to take on a special role when followed by a
> double-quote, in which case the backslash would be discarded while the
> double-quote would be interpreted as a character rather than the end of
> the string. Interpretation of other escape sequences was considered not
> a feature of string literals, but rather of user message stream output
> (`#debug` and its kin).
> 

There is a lot of history floating around. :)

> 
>> Significance is relevant. I have no idea how many people use B3D. And I
>> can't remember if Blender outputs the directory separator as a "\" or
>> "/". (I have stopped using it until some of the bug reports I have
>> submitted, have been fixed.)
> 
> Yeah, I guess B3D qualifies as relevant indeed.
> 
> 

Very pleased to hear it. :D

>> It is possible to get B3D to export bespoke code. So we could set a flag
>> telling the parser that the directory separator was "\".
>> e.g. windoze_separator = true
>> If that would help cut down the complexity.
> 
> Not sure I understand what you mean. Would you change B3D, or would you
> change POV-Ray?
> 
> When it comes to ideal format in POV-Ray scenes, it is highly
> recommended to always use "/" as path separator. Works on all platforms,
> 'Doze incldued.
> 

But not with MS-DOS 1.0. It didn’t support directories at all. The 
forward slash was used as a switch.

https://www.howtogeek.com/181774/why-windows-uses-backslashes-and-everything-else-uses-forward-slashes/


> 
>> Which reminds me. I use that facility to overwrite the version number
>> B3D exports by declaring it after the first one. Will that workaround
>> still work?
> 
> Uh... please elaborate. How would the resulting scene file look like?
> 

#version 3.6;


//------- Scene Raw Script Begin -------
#version 3.7 ;
//------- Scene Raw Script End ---------

[The rest of the scene...]

-- 

Regards
     Stephen


Post a reply to this message

<<< Previous 10 Messages Goto Latest 10 Messages Next 10 Messages >>>

Copyright 2003-2021 Persistence of Vision Raytracer Pty. Ltd.