Large amounts of dynamically placed objects, such as grass or trees, can
take a long time to parse.
Usually you would write these objects to a file, and then just use an
include statement to place them.

What if Pov Ray could handle this instead?
You would write something like this:

cache "grass"{
   #declare a=0;
   #while(a<100000)
     placeGrass(a)
     ...
   #end
}

Pov Ray will automatically take the results and and save it to a file
named grass.
Anytime the scene is rendered it will check if the file exists, if it 
doesn't the contents of the block will be parsed, otherwise the file 
will just be slotted in in place of the block.

Post a reply to this message

Am Wed, 15 Apr 2009 13:32:07 +0200 schrieb C:

> Large amounts of dynamically placed objects, such as grass or trees, can
> take a long time to parse.
> Usually you would write these objects to a file, and then just use an
> include statement to place them.
> 
> What if Pov Ray could handle this instead? You would write something
> like this:
> 
> cache "grass"{
>    #declare a=0;
>    #while(a<100000)
>      placeGrass(a)
>      ...
>    #end
> }
> 
> Pov Ray will automatically take the results and and save it to a file
> named grass.
> Anytime the scene is rendered it will check if the file exists, if it
> doesn't the contents of the block will be parsed, otherwise the file
> will just be slotted in in place of the block.
Actually the code should check if the definition of the cache block and 
any functions/prototypes used in it have changed since the cache was 
written. Else people would start complaining that their changes to code 
in chache blocks don't take effect ;)

I think there is a consensus that SDL 4 should be a bytecode-compiled 
language and as such it will execute much faster than SDL 3 did. Such a 
"cache" should be needed much less than now. But sometimes calculations 
are very time-consuming so I think there is some justification for that 
feature.
Historically for POV-Ray, plain SDL has been the only way to save scene 
data to disk. I believe that serialization of scene data would be a 
useful feature, but perhaps instead of developing just another binary 
scene format, POV-Ray 4 could just save compiled bytecode to recreate the 
scene or parts of it. Executing it should be almost as fast as loading a 
binary format while supporting all the complexity behind POV-Ray scenes 
perfectly. Perhaps POV-Ray could even do this caching automatically 
without an extra "cache" block.

Post a reply to this message

Lukas Winter escreveu:
> Am Wed, 15 Apr 2009 13:32:07 +0200 schrieb C:
> 
>> Large amounts of dynamically placed objects, such as grass or trees, can
>> take a long time to parse.
>> Usually you would write these objects to a file, and then just use an
>> include statement to place them.
>>
>> What if Pov Ray could handle this instead? You would write something
>> like this:
>>
>> cache "grass"{
>>    #declare a=0;
>>    #while(a<100000)
>>      placeGrass(a)
>>      ...
>>    #end
>> }
>>
>> Pov Ray will automatically take the results and and save it to a file
>> named grass.
>> Anytime the scene is rendered it will check if the file exists, if it
>> doesn't the contents of the block will be parsed, otherwise the file
>> will just be slotted in in place of the block.
> Actually the code should check if the definition of the cache block and 
> any functions/prototypes used in it have changed since the cache was 
> written. Else people would start complaining that their changes to code 
> in chache blocks don't take effect ;)
> 
> I think there is a consensus that SDL 4 should be a bytecode-compiled 
> language and as such it will execute much faster than SDL 3 did. Such a 
> "cache" should be needed much less than now. But sometimes calculations 
> are very time-consuming so I think there is some justification for that 
> feature.
> Historically for POV-Ray, plain SDL has been the only way to save scene 
> data to disk. I believe that serialization of scene data would be a 
> useful feature, but perhaps instead of developing just another binary 
> scene format, POV-Ray 4 could just save compiled bytecode to recreate the 
> scene or parts of it. Executing it should be almost as fast as loading a 
> binary format while supporting all the complexity behind POV-Ray scenes 
> perfectly. Perhaps POV-Ray could even do this caching automatically 
> without an extra "cache" block.

Now, I'm probably all wrong and messed up here so if anyone cares may 
correct me.  But I do feel JIT-compiled, bytecode compiled or even 
native code compiled SDL would do nothing for performance except perhaps 
make it worse should a more expressive and general language be used for 
the next SDL.

I've recently been taking a look at Povray beta source, in particular 
parse.hpp and associated.  It is the purpose of parse to create a long 
list of povray objects and it does that by analizing the SDL code, 
breaking it down into tokens and calling as it goes the internal 
functions which will create the objects, like Create_Sphere or Create_Plane.

Quite frankly, hardly any external general-purpose language would do 
much better at pure speed than using this method.  They'll still have to 
parse and create the objects in the same manner, by binding some of its 
particular calls to calls to povray object-creating functions and 
methods.  But there's also a compilation step.  And I don't believe 
parsing itself would be faster by going with a general purpose language, 
even lightweight ones like Lua, Tcl or Scheme.

How can you be faster for a general purpose language that would allow 
for creation of unrestricted user-level functions rather than pov's 
textual macros and still bring possibly much more unecessary stuff to 
the table?  How can that be faster than the SDL's straightforward 
parsing by directly calling the creation functions?

I was thinking even if I would use a native compiled language perhaps it 
would not be worth it because I believe most users of povray SDL enjoy 
an iteractive development style:  write a little, parse with low quality 
settings, go back, loop.  Would the time taken on parsing and 
compilation be worth it?  Current SDL only takes time on parsing, 
compilation merely means calling the internal creation functions.

I'm not against a better, more expressive SDL that would allow for far 
better and convenient scripting and do away with many name clashes with 
true scoping rules.  Just pointing out that that kind of power may also 
mean slower rather than higher performance.

-- 
a game sig: http://tinyurl.com/d3rxz9

Post a reply to this message

nemesis <nam### [at] gmailcom> wrote:
> Now, I'm probably all wrong and messed up here so if anyone cares may 
> correct me.  But I do feel JIT-compiled, bytecode compiled or even 
> native code compiled SDL would do nothing for performance except perhaps 
> make it worse should a more expressive and general language be used for 
> the next SDL.

  Rather than speculate, you could make actual tests.

  I once did that, out of curiosity. I made an ascii-mandelbrot generator
in both POV-Ray SDL and Perl (which is a byte-compiled, interpreted scripting
language). They were line-by-line almost identical to each other, just with
the correspondent syntax of each language. The Perl version was something
like 20 times faster than the SDL version.

-- 
                                                          - Warp

Post a reply to this message

Warp escreveu:
> nemesis <nam### [at] gmailcom> wrote:
>> Now, I'm probably all wrong and messed up here so if anyone cares may 
>> correct me.  But I do feel JIT-compiled, bytecode compiled or even 
>> native code compiled SDL would do nothing for performance except perhaps 
>> make it worse should a more expressive and general language be used for 
>> the next SDL.
> 
>   Rather than speculate, you could make actual tests.
> 
>   I once did that, out of curiosity. I made an ascii-mandelbrot generator
> in both POV-Ray SDL and Perl (which is a byte-compiled, interpreted scripting
> language). They were line-by-line almost identical to each other, just with
> the correspondent syntax of each language. The Perl version was something
> like 20 times faster than the SDL version.

That's unexpected.

 From just the looks of it, the SDL parser seems so straightforward that 
it's difficult to reason where any slowness could be coming from. 
Perhaps the textual macro expansion...

OTOH, you do not mean you integrated Perl inside povray, did you? 
Instead of trying to measure SDL's speed on something it's not 
specialized at, what about integrating Perl inside povray so it could 
directly call povray's object creation functions via FFI or something 
and then use that to parse a simple Perl script that would generate a 
scene?  And then benchemarking that against the builtin SDL.

What you did was just measure SDL's slow macro processing, not it's 
speed at povray internal object creation.

I'm more than willing to do that with some small Scheme implementation 
even if it proves in vain.  I'll dig deeper into it and will certainly 
make actual tests.

-- 
a game sig: http://tinyurl.com/d3rxz9

Post a reply to this message

Warp wrote:
> nemesis <nam### [at] gmailcom> wrote:
>> Now, I'm probably all wrong and messed up here so if anyone cares may 
>> correct me.  But I do feel JIT-compiled, bytecode compiled or even 
>> native code compiled SDL would do nothing for performance except perhaps 
>> make it worse should a more expressive and general language be used for 
>> the next SDL.
> 
>   Rather than speculate, you could make actual tests.
> 
>   I once did that, out of curiosity. I made an ascii-mandelbrot generator
> in both POV-Ray SDL and Perl (which is a byte-compiled, interpreted scripting
> language). They were line-by-line almost identical to each other, just with
> the correspondent syntax of each language. The Perl version was something
> like 20 times faster than the SDL version.
> 

Even if the new SDL is lightning fast when your placing 100 000 blades 
of grass dynamically, it's going to take a while.

Post a reply to this message

C <the### [at] gmailcom> wrote:
> cache "grass"{
>    #declare a=0;
>    #while(a<100000)
>      placeGrass(a)
>      ...
>    #end
> }
>
> Pov Ray will automatically take the results and and save it to a file
> named grass.

Nice one. I already thought about saving already-parsed code (or even scene
objects) to be "quick-loaded" later, so that scene scripts could store interim
stuff and reload it later on demand. But using a mechanism to *automatically*
decide whether to generate or load the code... that'd be cool indeed.

Of course some mechanism should be in place to ensure that the pre-generated
stuff still matches, so it can re-run the code if something has been changed.
But I guess that should be rather easy to ensure by storing along with the
compiled stuff (1) some hash value of the respective source code (whitespace-
and comment-"neutral"), and (2) a table of variables accessed by that code
section, hashes of their values at the start of the code, and the actual values
of any variables changed / created / reset by the code.

For more flexibility, the syntax could be extended to include a version value,
which would be checked instead of the hash and all the variable values, so that
the code would only be re-run if the user updated that value.

Post a reply to this message

Lukas Winter <web### [at] geloeschtremovethisandthedotbeforenet> wrote:
> Historically for POV-Ray, plain SDL has been the only way to save scene
> data to disk. I believe that serialization of scene data would be a
> useful feature, but perhaps instead of developing just another binary
> scene format, POV-Ray 4 could just save compiled bytecode to recreate the
> scene or parts of it.

Yup. I guess that's how I'd too it as well.

Instantiating the items isn't that time consuming after all (especially in case
of already-parsed bytecode); it's usually the user code deciding where to place
the things in the first place.

Saving the current scene contents by generating bytecode to re-create it also
saves a lot of brain-wrecking how to actually serialize the data; after all, I
guess we want saved scenes to be portable between different platforms. I do,
for instance: I use both Linux and Windows, and occasionally exchange some
auto-generated .inc files between them. POV 4.0 should be able to do that at
least as easy.

And besides, I really expect not much of a performance loss when using bytecode
instead of a portable serialization framework. And it might be easier to cope
with compatibility stuff regarding new versions.

So yes, "serialization" by generating bytecode to re-create the current "scene
element tree" would be the way to go, if I'm asked.

Post a reply to this message

nemesis <nam### [at] gmailcom> wrote:
> Quite frankly, hardly any external general-purpose language would do
> much better at pure speed than using this method.  They'll still have to
> parse and create the objects in the same manner, by binding some of its
> particular calls to calls to povray object-creating functions and
> methods.  But there's also a compilation step.  And I don't believe
> parsing itself would be faster by going with a general purpose language,
> even lightweight ones like Lua, Tcl or Scheme.

There is a difference:

Bytecode is more compact. It doesn't have whitespace, nor comments. It doesn't
refer to variables or keywords (commands in bytecode) by names, but by indices
in a table instead. Its command block ends are already identified. Its code -
and moreover that of include files - is executed from memory instead of a
(hopefully buffered) disk access module.

So...

- Disk access is reduced to the absolute minimum, not only because the files are
likely smaller, but also because macro calls don't cause re-opening of the files
they're declared in.

- No need to seek over whitespace and comments.

- No need to seek the end of a token.

- No need to re-compute the hash of a variable name each time it is encountered.

- No need to seek the end of an else-block.

- No need to re-load code of macros executed.

All this does not take up *much* time per statement - but if for instance you're
calling the VRand macro in some loop a million times, then you *will* notice the
difference.

> I'm not against a better, more expressive SDL that would allow for far
> better and convenient scripting and do away with many name clashes with
> true scoping rules.  Just pointing out that that kind of power may also
> mean slower rather than higher performance.

It will possibly be slower indeed for scripts that execute comparatively fast
and make not much use of include files. But it will sure make a difference for
those scenes that currently take half an hour of parsing, whiling away in some
loop forcing that VRand macro to be re-loaded and re-parsed for the gazillionth
time, character by character.

Post a reply to this message

nemesis <nam### [at] gmailcom> wrote:
> OTOH, you do not mean you integrated Perl inside povray, did you?
> Instead of trying to measure SDL's speed on something it's not
> specialized at, what about integrating Perl inside povray so it could
> directly call povray's object creation functions via FFI or something
> and then use that to parse a simple Perl script that would generate a
> scene?  And then benchemarking that against the builtin SDL.
>
> What you did was just measure SDL's slow macro processing, not it's
> speed at povray internal object creation.

That's a valid benchmark nonetheless: My slowest-parsing POV scenes spend most
of their time doing math, often not even instantiating objects directly but
instead doing text output - generating .inc files that I can later just include
instead of running the time-consuming parts again.

On some occasions, it's 30 minutes of math vs. 5 minutes of actual scene stuff
generation.

Post a reply to this message