hi,

attached include file provides a macro for reading and writing the POV-Ray
variant of CSV data files.  the use is documented in a HTML page, and a few
demo/example scenes.


enjoy, jr.

Post a reply to this message

Attachments:
Download 'filed.zip' (26 KB)

On 10/13/21 10:43 AM, jr wrote:
> hi,
> 
> attached include file provides a macro for reading and writing the POV-Ray
> variant of CSV data files.  the use is documented in a HTML page, and a few
> demo/example scenes.
> 
> 

Great work! Another useful macro - and some cool demos too. Robot 
battles... ;-)

Ran through all the demos and didn't pick up any issues with my current 
package of IO fixes / changes. I do have some comments on the Filed() 
package itself.

---

"...missing out a field or an empty field..."

perhaps?

"...a missing field or an empty field..."

---

The flowers.pov file already has the tabulated.inc guard set to 1 
contrary to the scene's comments.

---

The flowers_read.pov animation doesn't look to me to be picking up all 
the yellow flowers? Maybe one or two per asteroid? I didn't investigate 
though, perhaps it's working as expected?

---

With the flowers.ini AA, at least on linux with v3.8 and povr, the 
method 3 AA doesn't look very good. Perhaps something like:

sampling_method = 2
antialias_threshold = 0.005
antialias_depth = 3
jitter = on

---

The toa example needs too the jr's ruled.inc - Ruled() and this is not 
mentioned in the documentation and there is no pointer as to where to 
grab it.

---

The toa example needs the arial.ttf font which is not - by default - 
available for most non windows users.

---

With the toa example, where the "toa_state.txt" or "toa_run.log" already 
exist we get an error message via toal__chkFilesExist(), but the error 
message only says "existing file(s)" were found.

---

Bill P.

Post a reply to this message

Hi Bill,
I shall just answer on those issues you mention about flowers.

Op 14/10/2021 om 19:38 schreef William F Pokorny:
> On 10/13/21 10:43 AM, jr wrote:
>> hi,
>>
>> attached include file provides a macro for reading and writing the 
>> POV-Ray
>> variant of CSV data files.  the use is documented in a HTML page, and 
>> a few
>> demo/example scenes.
>>
>>
> 
> Great work! Another useful macro - and some cool demos too. Robot 
> battles... ;-)
> 
> Ran through all the demos and didn't pick up any issues with my current 
> package of IO fixes / changes. I do have some comments on the Filed() 
> package itself.
> 
> ---
> 
> "...missing out a field or an empty field..."
> 
> perhaps?
> 
> "...a missing field or an empty field..."
> 
> ---
> 
> The flowers.pov file already has the tabulated.inc guard set to 1 
> contrary to the scene's comments.
> 
I missed that one!

> ---
> 
> The flowers_read.pov animation doesn't look to me to be picking up all 
> the yellow flowers? Maybe one or two per asteroid? I didn't investigate 
> though, perhaps it's working as expected?
> 
If you kept the default numbers of flowers (400 Red; 300 Yellow; total 
of 700 flowers in flowers.txt), those numbers are also rendered in 
flowers_read.pov. At least, that is what I see here. To be certain, just 
delete flowers.txt, and run flowers.pov again, after checking the number 
of flowers to be planted.

However, if you run the flowers.ini animation, be sure to set 
initial_frame and final_frame to the values you want. Standard, a range 
of frames is given, typically showing the passage from the Red flowers 
to the Yellow ones (in the default settings). For a complete animation, 
both values of course should be set accordingly.

> ---
> 
> With the flowers.ini AA, at least on linux with v3.8 and povr, the 
> method 3 AA doesn't look very good. Perhaps something like:
> 
> sampling_method = 2
> antialias_threshold = 0.005
> antialias_depth = 3
> jitter = on
> 
These are jr's settings. Note that I had to comment out the file_gamma = 
1 line, as this command resulted in very dark frames, compared to the 
original renders.

> ---
> 
> The toa example needs too the jr's ruled.inc - Ruled() and this is not 
> mentioned in the documentation and there is no pointer as to where to 
> grab it.
> 
> ---
> 
> The toa example needs the arial.ttf font which is not - by default - 
> available for most non windows users.
> 
> ---
> 
> With the toa example, where the "toa_state.txt" or "toa_run.log" already 
> exist we get an error message via toal__chkFilesExist(), but the error 
> message only says "existing file(s)" were found.
> 
> ---
> 
> Bill P.


-- 
Thomas

Post a reply to this message

hi,

William F Pokorny <ano### [at] anonymousorg> wrote:
> On 10/13/21 10:43 AM, jr wrote:
> > attached include file provides a macro for reading and writing the POV-Ray
> > variant of CSV data files.  the use is documented in a HTML page, and a few
> > demo/example scenes.
>
> Great work! Another useful macro - and some cool demos too. Robot
> battles... ;-)

thank you.


> Ran through all the demos and didn't pick up any issues with my current
> package of IO fixes / changes. I do have some comments on the Filed()
> package itself.

nice.  (and taken time for feedback.  \o/ yay..  :-)  in order, then)


> ---
>
> "...missing out a field or an empty field..."
>
> perhaps?
>
> "...a missing field or an empty field..."

agree.  will make the change.


> ---
>
> The flowers.pov file already has the tabulated.inc guard set to 1
> contrary to the scene's comments.

Thomas held up his hand, but I did not "see" it either.  will change the
default, and thinking of adding an '#if' guard in 'flowers_read.pov' too.


> ---
>
> The flowers_read.pov animation doesn't look to me to be picking up all
> the yellow flowers? Maybe one or two per asteroid? I didn't investigate
> though, perhaps it's working as expected?

the animation runs from planting the last few reds to the first few dozen
yellows only.


> ---
>
> With the flowers.ini AA, at least on linux with v3.8 and povr, the
> method 3 AA doesn't look very good. Perhaps something like:
>
> sampling_method = 2
> antialias_threshold = 0.005
> antialias_depth = 3
> jitter = on

as Thomas wrote, setting the 'file_gamma' in the ini was a bad idea, will remove
it.  does removing it address the issue without changing 'sampling_method' etc?


> ---
>
> The toa example needs too the jr's ruled.inc - Ruled() and this is not
> mentioned in the documentation and there is no pointer as to where to
> grab it.

yes.  will add a 'references' section to the html, for both 'ruled.inc' and
'tabulated.inc'.


> ---
>
> The toa example needs the arial.ttf font which is not - by default -
> available for most non windows users.

good point too.  will switch to a distribution font.


> ---
>
> With the toa example, where the "toa_state.txt" or "toa_run.log" already
> exist we get an error message via toal__chkFilesExist(), but the error
> message only says "existing file(s)" were found.

or, conversely, "missing file(s)".  my thinking was/is that for animation run, I
only need to know whether both or neither (state + log) exist, depending on
frame_number, so the macro only provides a result for that.  how should I write
the corresponding messages to read better?


regards, jr.

Post a reply to this message

On 10/15/21 5:14 AM, jr wrote:
> hi,
> 
...
> 
>> ---
>>
>> The flowers.pov file already has the tabulated.inc guard set to 1
>> contrary to the scene's comments.
> 
> Thomas held up his hand, but I did not "see" it either.  will change the
> default, and thinking of adding an '#if' guard in 'flowers_read.pov' too.
> 

Adding it too flowers_read.pov would be helpful. Aiding my confusion as 
to how the animation was supposed to work was the table being in the 
background showing about half red(1) entries and yellow entries - which 
I rather dumbly took as some indication of what I should see for 
distribution for flower colors.

(1) The initial set of flowers end up orange. Maybe?

#declare AmericanRose = srgb <1,0.01176,0.24314>;
...
pigment {AmericanRose*0.5}


> 
>> ---
>>
>> The flowers_read.pov animation doesn't look to me to be picking up all
>> the yellow flowers? Maybe one or two per asteroid? I didn't investigate
>> though, perhaps it's working as expected?
> 
> the animation runs from planting the last few reds to the first few dozen
> yellows only.
> 

Thanks Thomas, I now understand.

> 
>> ---
>>
>> With the flowers.ini AA, at least on linux with v3.8 and povr, the
>> method 3 AA doesn't look very good. Perhaps something like:
>>
>> sampling_method = 2
>> antialias_threshold = 0.005
>> antialias_depth = 3
>> jitter = on
> 
> as Thomas wrote, setting the 'file_gamma' in the ini was a bad idea, will remove
> it.  does removing it address the issue without changing 'sampling_method' etc?
> 

I dug a little. The ugly background table AA comes from using 
'antialias_gamma = 1' though the file_gamma at 1.0 makes the saved 
images linear/dark too. Both settings should go away.

Of interest, perhaps, regarding aa method 3 here. I measured these times 
and results on my Ubuntu 20.04 system.

  8.16cpu 0:03.16elapsed (antialias_gamma = 1; shipped result ugly)
26.76cpu 0:07.89elapsed (No antialias_gamma = 1; result good)
19.31cpu 0:06.10elapsed (No antialias_gamma = 1; result good; povr(*))

(*) Here the development compile being used is not that optimized and 
it's still faster. This is demonstrating the povr branch's fix to 
prevent the color channel sqrt() of negative values and subsequent 
undefined behavior. With povr the method 3 AA isn't sometimes running 
exhaustively to the recursion limit no matter other settings.

First, I screwed up on my recommendation in not have the 
antialias_threshold at 0.0. The fastest good result I see for the 
suggested render sizes, if that's your aim when shipping the package, is 
to use:

sampling_method = 2
antialias_threshold = 0.0
antialias_depth = 2
jitter = off
// 10.85user 0.09system 0:03.86elapsed

With the background table we have a very regular grid and this a case. 
Ultimately, we have to shoot enough rays to 'see' the detail we want 
cleanly visible.

The method 3 AA often works, but it presently has a known weakness in 
not being able to force an initial minimum set of samples (and this 
should probably be done to some isotropic density of samples).

The faster method 2 approach above might miss detail too at other render 
resolutions / aspect ratios.

...
> 
>> ---
>>
>> With the toa example, where the "toa_state.txt" or "toa_run.log" already
>> exist we get an error message via toal__chkFilesExist(), but the error
>> message only says "existing file(s)" were found.
> 
> or, conversely, "missing file(s)".  my thinking was/is that for animation run, I
> only need to know whether both or neither (state + log) exist, depending on
> frame_number, so the macro only provides a result for that.  how should I write
> the corresponding messages to read better?
> 

I'm a fan of seeing the actual file names if relatively easy, but at a 
minimum maybe something like "existing output files()" were found. 
(Missing input...?). Maybe the issue not that important with other 
include sorted.

My initial animation render didn't work because I was missing an 
include. That first attempt did create those two output files, but I 
didn't notice. And because I'd played around with the earlier demos I 
first thought perhaps I'd created some file which was colliding with the 
toa demo. I was slow to realize I had created files on the first failed 
attempt which were now an issue. If everything is cleaner from the 
start, there will be less potential for confusion.

Bill P.

Post a reply to this message

Op 15/10/2021 om 16:46 schreef William F Pokorny:

> (1) The initial set of flowers end up orange. Maybe?
> 
> #declare AmericanRose = srgb <1,0.01176,0.24314>;
> ...
> pigment {AmericanRose*0.5}
> 

That is a beautiful colour! I added it to my collection of 
textures/pigments (under construction).


-- 
Thomas

Post a reply to this message

Attachments:
Download 'wfp_textures_americanrose.jpg' (48 KB)

Preview of image 'wfp_textures_americanrose.jpg'

hi,

William F Pokorny <ano### [at] anonymousorg> wrote:
> ...
> I dug a little. The ugly background table AA comes from using
> 'antialias_gamma = 1' though the file_gamma at 1.0 makes the saved
> images linear/dark too. Both settings should go away.
> ...
> First, I screwed up on my recommendation in not have the
> antialias_threshold at 0.0. The fastest good result I see for the
> suggested render sizes, if that's your aim when shipping the package, is
> to use:
>
> sampling_method = 2
> antialias_threshold = 0.0
> antialias_depth = 2
> jitter = off
> // 10.85user 0.09system 0:03.86elapsed

thank you for "digging" and recommending a better set of parameters.  will
simply use those.


> ... That first attempt did create those two output files, but I
> didn't notice. ...

:-)  lost count of the number of times this happened to me, in the later stages
of development.


regards, jr.

Post a reply to this message

"jr" <cre### [at] gmailcom> wrote:
> attached include file provides a macro for reading and writing the POV-Ray
> variant of CSV data files.  the use is documented in a HTML page, and a few
> demo/example scenes.

version 2.  includes the various improvements to documentation and demo code
suggested by WFP (thanks!).


enjoy, jr.

Post a reply to this message

Attachments:
Download 'filed2.zip' (27 KB)

hi,

William F Pokorny <ano### [at] anonymousorg> wrote:
> ...
> antialias_threshold = 0.0

adopted the settings as suggested.  see a little over 3 seconds improvement for
'toa'.  :-)


> ...
> I'm a fan of seeing the actual file names if relatively easy, but at a
> minimum maybe something like "existing output files()" were found.
> (Missing input...?). Maybe the issue not that important with other
> include sorted.

not easy, hope the compromise will suit.  have posted an updated version
incorporating all suggestions a moment ago.


regards, jr.

Post a reply to this message

"jr" <cre### [at] gmailcom> wrote:
> "jr" <cre### [at] gmailcom> wrote:
> > attached include file provides a macro for reading and writing the POV-Ray
> > variant of CSV data files.  the use is documented in a HTML page, and a few
> > demo/example scenes.
>
> version 2.  includes the various improvements to documentation and demo code
> suggested by WFP (thanks!).

after running some tests with larger files, I have added the following paragraph
to the HTML page:

"using the Filed() macro comes at a run-time cost, alas.  for data sets of
hundreds, or low thousands, of records, the time taken is unnoticeable, though
if there are hundreds of thousands or more, it may (in an animation, perhaps)
become enough of an issue to consider sticking with the #read and #write
directives."

with 100k sphere centre and radius records for instance, POV-Ray needs around
one second for the writing and the same for the read back.  Filed() comes in at
around eight seconds write and ten seconds read.


regards, jr.

Post a reply to this message

On 10/26/21 9:57 AM, jr wrote:
> "jr" <cre### [at] gmailcom> wrote:
...
> 
> with 100k sphere centre and radius records for instance, POV-Ray needs around
> one second for the writing and the same for the read back.  Filed() comes in at
> around eight seconds write and ten seconds read.
> 

Hi,

I'm a little surprised at the magnitude of the slow down. Are you timing 
with a v3.8 beta or something else? I'm expecting the files are all on a 
ramdisk?

Thinking aloud...

If a povr branch, was it heavily optimized ahead of the build/make via 
configure script options?

Re: The povr branch is running with a C++11 string hash function for the 
symbol tables in the parser / vm over the traditional POV-Ray one. In my 
testing the C++11 one is only faster if the compile options are 
relatively aggressive(1,2).

In general, with the dictionary stuff being relatively new to v3.8, 
there is likely room to make the parser faster around that functionality.

I'd be willing to play a little with your 100K spheres case - if you 
want?

Bill P.

(1) - A reminder, the povr branch build system does NO optimization by 
default when the configure script is run.

(2) - The C++11 hashing of strings is much better, but it's more complex 
code and takes longer than POV-Ray's method. The old hashing tends to 
clump table entries around the leading ascii character value(s).

Post a reply to this message

hi,

William F Pokorny <ano### [at] anonymousorg> wrote:
> On 10/26/21 9:57 AM, jr wrote:
> > with 100k sphere centre and radius records for instance, POV-Ray needs around
> > one second for the writing and the same for the read back.  Filed() comes in at
> > around eight seconds write and ten seconds read.
> >
>
> I'm a little surprised at the magnitude of the slow down. Are you timing
> with a v3.8 beta or something else? I'm expecting the files are all on a
> ramdisk?

the reading did not surprise much, every item returned is "handled" three times,
but the (lack of) write performance I do not understand.


> Thinking aloud...
>
> If a povr branch, ...

ah, no, "stock" alpha.9945627.  fyi (snipped the context below), all POV-Rays
and povr etc built with '-O2'.


> I'd be willing to play a little with your 100K spheres case - if you
> want?

thanks, sure.  have attached a reader and writer scene.  for the writer can use
'declare=N=number' to create number records, for both scenes can use
'declare=FM=true' to use Filed() instead of "raw" directives.


regards, jr.

Post a reply to this message

Attachments:
Download 'wfp_filed.tar.xz.dat' (1 KB)

On 10/28/21 3:02 AM, jr wrote:
...
> 
> the reading did not surprise much, every item returned is "handled" three times,
> but the (lack of) write performance I do not understand.
...
>> I'd be willing to play a little with your 100K spheres case - if you
>> want?
> 
> thanks, sure.  have attached a reader and writer scene.  for the writer can use
> 'declare=N=number' to create number records, for both scenes can use
> 'declare=FM=true' to use Filed() instead of "raw" directives.
...

Thank you. I've grabbed it. I'll need my morning coffee too! :-)

Thinking I'll do some real profiling again. It's gotten to be a year or 
more since I've done any - and I'd like to look at rtr that way too.

With rtr I can tell some of the delay with smaller render sizes is the 
mutex locking mechanism itself. This aligning with a note I'd guess 
Chris made way back when in the code. Beyond that I'm not sure what 
profiling might reveal - or not.

Bill P.

Post a reply to this message

On 10/28/21 6:11 AM, William F Pokorny wrote:
> On 10/28/21 3:02 AM, jr wrote:
...
> 
> Thank you. I've grabbed it. I'll need my morning coffee too! :-)
> 
...

Not yet to profiling, but I've turned up a few things. First, I don't 
see the +700% write side slow down you see, but rather about +280% so 
this is more in line with your expectations for write vs read.

I'm interested in where povr is and I was surprised to find it slower 
than p380b1.

On investigation found the parser updates coming after what is in the 
v3.8 betas(1) - which povr adopted / branched from - are themselves 
slower. More significantly so on the write side. My povr branch is 
running faster than the branch point, but... Guess for me that is the 
first thing to try and figure out.

Bill P.

(1) - Christoph backtracked the newest parser updates for the v3.8 release.

Ref:

stress_wr.pov  (100k)
p380   raw - 2.05user 0.25system 0:02.84elapsed 81%CPU
p380   fld - 6.87user 0.33system 0:07.74elapsed 93%CPU

p380b1 raw - 1.57user 0.02system 0:02.14elapsed 74%CPU
p380b1 fld - 6.02user 0.06system 0:06.61elapsed 91%CPU

povr   raw - 1.92user 0.24system 0:02.71elapsed 79%CPU
povr   fld - 6.39user 0.28system 0:07.21elapsed 92%CPU

p380b1 raw ---> fld  1.57 -> 6.02 ---> +283.44%  (You suggested +700%)
p380   raw ---> fld  2.05 -> 6.97 ---> +240.00%
povr   raw ---> fld  1.92 -> 6.39 ---> +232.81%

p380b1 -> p380 raw  1.57 -> 2.05 ---> +30.57% (p380 povr branch point)
p380   -> povr raw  2.05 -> 1.92 ---> -6.34%  (povr < 380 branch point)

p380b1 -> p380 fld  6.02 -> 6.87 ---> +14.12% (p380 povr branch point)
p380   -> povr fld  6.87 -> 6.39 --->  -6.99% (povr < 380 branch point)


stress_rd.pov  (100k)
p380   raw - 0.64user 0.02system 0:01.20elapsed 55%CPU
p380   fld - 8.39user 0.08system 0:09.00elapsed 94%CPU

p380b1 raw - 0.62user 0.03system 0:01.19elapsed 54%CPU
p380b1 fld - 7.69user 0.07system 0:08.29elapsed 93%CPU

povr   raw - 0.61user 0.01system 0:01.17elapsed 53%CPU
povr   fld - 7.90user 0.06system 0:08.50elapsed 93%CPU

p380b1 raw ---> fld  0.62 -> 7.69 ---> +1140.32%
p380   raw ---> fld  0.64 -> 8.39 ---> +1210.94%
povr   raw ---> fld  0.61 -> 7.90 ---> +1195.08%

p380b1 -> p380 raw  0.62 -> 0.64 ---> +3.23% (p380 povr branch point)
p380   -> povr raw  0.64 -> 0.61 ---> -4.69% (povr < 380 branch point)

p380b1 -> p380 fld  7.69 -> 8.39 ---> +9.10% (p380 povr branch point)
p380   -> povr fld  8.39 -> 7.90 ---> -5.84% (povr < 380 branch point)

Post a reply to this message

hi,

William F Pokorny <ano### [at] anonymousorg> wrote:
> ...
> Not yet to profiling, but I've turned up a few things. First, I don't
> see the +700% write side slow down you see, but rather about +280% so
> this is more in line with your expectations for write vs read.

intend to install the new povr in the coming days.  will then run some tests,
including another 100k.  thanks for posting the timings.  (1200% is a high price
to pay, for convenience..  :-))


regards, jr.

Post a reply to this message

On 10/29/21 5:35 AM, jr wrote:
> hi,
> 
> William F Pokorny <ano### [at] anonymousorg> wrote:
>> ...
>> Not yet to profiling, but I've turned up a few things. First, I don't
>> see the +700% write side slow down you see, but rather about +280% so
>> this is more in line with your expectations for write vs read.
> 
> intend to install the new povr in the coming days.  will then run some tests,
> including another 100k.  thanks for posting the timings.  (1200% is a high price
> to pay, for convenience..  :-))
>

Ultimately it's user time which matters. If machines are paying and 
users still win measuring task time, it's the right trade off.

---
I understand more as of this morning. A large chunk of the p380 branch / 
povr slow down was due always-active soft asserts Christoph added to 
some of the last parser updates. These took me a while to understand and 
untangle/turn off for the 'regular' povr build.

So! Timing with my in hand povr.

stress_wr.pov
p380b1     ---> povr fld  6.87 -> 5.42 ---> -21.11%

p380b1 raw ---> fld +283.44%
povr   raw ---> fld +234.57% (-40%)


stress_rd.pov
p380b1     ---> povr fld  8.39 -> 6.38 ---> -23.98%

p380b1 raw ---> fld +1140.32%
povr   raw ---> fld +1103.77% (-36%)

So, I'm happy to see povr and likely P380 too, without the soft asserts, 
is quite a bit faster. I expect some of the speed up is due Christoph's 
newer parser updates and some due my changes, but I've not done the p380 
builds and timings to see how the savings break out - I'm not all that 
interested so that work probably won't happen.

A puzzle I've not been able to run down is the larger 'system' write 
times post v3.8 beta. It's very roughly around +12.5% of the raw and fld 
write slow down v3.8 beta to p380 (so it hits povr too though it's 
faster overall). Something is driving up that system time from raw and 
fld writes, but I've not been able to find it.

For now I'm walking away from it - Christoph cleaned up / changed quite 
a lot related to file and text streams over the last months ahead of the 
branch point for povr. I don't see an obvious cause - and my 
guesses(shots in the dark) as to cause have missed.

I hope to get to some actual code profiling today.

Bill P.

Post a reply to this message

On 10/29/21 9:36 AM, William F Pokorny wrote:
> A puzzle I've not been able to run down is the larger 'system' write 
> times post v3.8 beta. It's very roughly around +12.5% of the raw and fld 
> write slow down v3.8 beta to p380 (so it hits povr too though it's 
> faster overall). Something is driving up that system time from raw and 
> fld writes, but I've not been able to find it.

An update.

--- stress_wr.pov
We can improve(1) the write side cpu time by -7.0% and the elapsed time 
by -9.3% by not using rand.inc's (VRand(rng_)) macro, but rather:

#local arr_[i_] = array mixed [2] \ 
{<rand(rng_),rand(rng_),rand(rng_)>,rand(rng_)};

Further, it appears the macro caching of VRand() is somehow tangled in 
the increased system time v3.8 beta to p380 branch/povr - I still don't 
see how. It still might be that if that cause could be run down, one 
could use VRand() without as much penalty.

(1) - To speed compiles I dropped the link time optimization. The 
improvement where configuring with --enable-lto will be somewhat different.

--- stress_rd.pov

A big chunk of the time is in the scanner.cpp code qualify bits and 
pieces of potential tokens. As to how to make that all 'significantly' 
faster within the current framework - no luck thus far. We'll see.

Bill P.

Post a reply to this message

On 10/30/21 7:10 PM, William F Pokorny wrote:
> A big chunk of the time is in the scanner.cpp code qualify bits and 
> pieces of potential tokens. As to how to make that all 'significantly' 
> faster within the current framework - no luck thus far. We'll see.

OK. I chased one pocket of scanner expense by changing from a set of 
conditional tests for character classification to a boolean array[256] 
with predetermined answers. I wanted to see how much movement / 
improvement I could get. It was a gain of about 3% on the write side and 
%2 percent on the read.

I think I see my way clear to folding up to eight character 
classifications in the space the boolean array is actually taking. This 
would allow us to change some(all?) of the 'if else if... else' 
conditional chains to a more direct switch construct. A WILD guess is 
we'd at most gain 10-15% total (2/3% above being in that savings).

If the guess about right, it would not be  a game changer - but the 
relative ease of implementation is there. I haven't decided whether to 
attempt the change in my povr branch.

Random thinking.
---------------

- In the end we are working character by character in the parser and 
this is expensive. Further, we are repeatedly classifying characters and 
tokens.

- The non-pure ascii (<=255) character encoding costs. An advantage utf8 
encoding for the SDL has over others is it's smaller and so faster.

- Relatedly, Christoph is using an internal to POV-Ray utf8 class based 
upon std::string. This is common practice with c++ programs and it's 
safe. It's the case though that as we walk that utf8 class by character, 
incrementing pointers the length checking is costing a few percent of 
the overall run time. Well... at least the profiling indicates this 
length checking cost.

- My updates above are related to the newest version of the parser and 
not the older one in v3.8 beta*.

- The newer parser changes in v4.0 and povr are substantial and they 
were only the initial push for what Christoph ultimately had in mind. 
Testing of those changes showed improved performance. I'm not, however, 
sure what all else was intended. Heck, to a degree the new parser is 
still new-ish to me. Until now, I've only dug around in the parser code 
to try and fix particular bugs.

- On staring at the profile results now over some days, I have few wild 
parser ideas banging around in my head for the parser. Unfortunately, 
most all are not that easy to just code up and try.

- A reminder the v4.0 / povr parsing is only faster than v3.8 beta* for 
the test cases in this thread - if a collection of parser asserts is 
turned off (see configparser.h). Christoph was actively working on the 
parser in late 2018 and early 2019. These would have eventually been 
turned off for normal release compiles.

Bill P.

Post a reply to this message

hi,

William F Pokorny <ano### [at] anonymousorg> wrote:
> ... It was a gain of about 3% on the write side and
> %2 percent on the read.
> ...

faster parsing is better of course, but I now think I need to do the read side
different.


regards, jr.

Post a reply to this message

hi,

'filed.inc' has been updated with a new '.Verbatim' key, inspired by the recent
"animation display window does not reset" thread.  example use below.

<https://drive.google.com/file/d/10pGH0yi_-8aBTQvTwQPB4AdfRl9JsTGg/view?usp=sharing>


enjoy, jr.

-----<snip>-----
#version 3.8;

global_settings {assumed_gamma 1}
box {0,1}

#declare fild_workingDir = "/tmp/";

#include "filed.inc"

#declare D = dictionary {
  .File     : "bash1.sh",
  .Access   : "write",
  .Fields   : array [1] {"S"},
  .Data     : array [3] {
                "echo \\\"new feature\\\" '.Verbatim' key,",
                "echo *thanks* JB :').'\necho $(date)",
                "ls --hide=systemd* ; echo"
              },
  .Strict   : false,
  .Verbatim : true,
  .Verbose  : true
};

Filed(D)

Post a reply to this message