Brent Fraser wrote:

> height_field(s) and pigment image_maps.  Most (90+% of 5 minutes) of the 
> "parse" time for each frame is spent loading the 7039x7402 tga height 
> field and the 14825x15587 png image map.

as an aside: Do you have enough memory to handle these files?
4.5 minutes sounds like a long time, maybe there is some swapping
going on. You might also experiment with different file formats.
And should you have 3+GB of memory available you might use 1GB
for a RAM disk.

Of course, not having to reload the image for each frame
would be best ;)

Post a reply to this message

Brent Fraser wrote:
>   I see that rtr/clockless animation in 3.7 will not re-parse, so the 5
>   minute file loading hit is done only once.  Now I need to figure out
>   where to add the file-saving code.  Is there a technical reason (i.e.
>   thread complexity) or a more administrative reason why that capability
>   is not in the current 3.7 beta?  Any pointers to where I could add it
>   would be appreciated.
> 
>   The problem with using the clockless animation approach to solve my
>   performance problem is that my script does actually contain a minor
>   amount of animation so I'd like to preserve it if possible (which
>   clockless animation does not currently allow).

The reason why the current clockless animation is even possible is *because*
it lacks the flexibility you're now needing...

I think instead of storing a number, it stores some special object saying it
has to animate that number. When the first frame is done, it updates that
one number and re-renders.

If you want to run any more complex code to move the camera, it will have to
re-parse to run the code again...

Post a reply to this message

Christian Froeschlin <chr### [at] chrfrde> wrote:
> And should you have 3+GB of memory available you might use 1GB
> for a RAM disk.

Weird. I always thought that using RAM disks as swap doesn't work (or, rather,
doesn't provide additional benefits).

-Mike

Post a reply to this message

SharkD wrote:
> Christian Froeschlin <chr### [at] chrfrde> wrote:
>> And should you have 3+GB of memory available you might use 1GB
>> for a RAM disk.
> 
> Weird. I always thought that using RAM disks as swap doesn't work (or, rather,
> doesn't provide additional benefits).
> 
> -Mike
> 

I think SharkD's suggestion was to use a RAM disk as an alternate location for the
tga/png files.  Unfortunately, the computers I'm using typically only have 1.5 to 2
gb, and usually all the RAM is used (along with some swap) in the rendering process.

More Background:
----------------
  For the rendering of this particular fly-thru movie, we distribute ("by hand") 15
.pov scripts between 6 to 10 computers.  Each script produces around 150 frames (which
we then assemble into a 15 fps movie).  These computers take about 2 to 3 days to
render all the frames.  Adding more RAM to the computers may help, but I'd like to
investigate changes to POV-Ray first.  

  I think only a few of the scripts contain object animation; that's why clockless
animation is so appealing.  If I can get an order of magnitude performance increase on
the 10 to 12 scripts that only move the camera, it would be great!

Thanks!
Brent Fraser

Post a reply to this message

Brent Fraser <bfr### [at] geoanalyticcom> wrote:
>   For the rendering of this particular fly-thru movie, we distribute ("by hand") 15
.pov scripts between 6 to 10 comp
uters.  Each script produces around 150 frames (which we then assemble into a 15 fps
movie).  These computers take abou
t 2 to 3 days to render all the frames.  Adding more RAM to the computers may help,
but I'd like to investigate changes
 to POV-Ray first.

If there were any alternative render farm options (see the "Render Farms"
thread), I would suggest them instead. But there aren't at the moment.

-Mike

Post a reply to this message

Nicolas Alvarez wrote:
> Brent Fraser wrote:
>>   I see that rtr/clockless animation in 3.7 will not re-parse, so the 5
>>   minute file loading hit is done only once.  Now I need to figure out
>>   where to add the file-saving code.  Is there a technical reason (i.e.
>>   thread complexity) or a more administrative reason why that capability
>>   is not in the current 3.7 beta?  Any pointers to where I could add it
>>   would be appreciated.
>>
>>   The problem with using the clockless animation approach to solve my
>>   performance problem is that my script does actually contain a minor
>>   amount of animation so I'd like to preserve it if possible (which
>>   clockless animation does not currently allow).
> 
> The reason why the current clockless animation is even possible is *because*
> it lacks the flexibility you're now needing...
> 
> I think instead of storing a number, it stores some special object saying it
> has to animate that number. When the first frame is done, it updates that
> one number and re-renders.
> 
> If you want to run any more complex code to move the camera, it will have to
> re-parse to run the code again...
> 

  I'm new to Pov-Ray so I'm still trying to get up to speed on the design philosophy. 
My understanding so far: the Pov files describe a scene containing objects with
properties and a viewpoint (camera).  Some properties change over time, driven by
Pov's CLOCK variable.  Since the properties are complex (can interact, etc) the scene
needs to be reconstructed (object properties re-calculated, possibly including the
camera postiion/orientation) for each frame (tick of the clock), and this is done by
re-parsing.  I get that; what I don't get is having to re-read height_field and
image_map files (non-changing raster files) on every frame render.

  We're likely using POV-Ray in a non-standard way: we want to render lots of small
frames (1024x768) by moving the camera over a small set of huge height fields (with
huge image maps) to make a movie.  From what I've read, POV's strength is rendering a
single large frame from a scene containing lots of objects; kind of the opposite of
our situation.  If the object definitions are small, then re-reading them on every
parse is a very minor part of the render time, but that's not our case.

   And I found it frustrating to read about MegaPov's "persistence of variables" only
to find out the feature is no longer available.

Thanks,
Brent Fraser

Post a reply to this message

Brent Fraser wrote:

> this is done by re-parsing.  I get that; what I don't get is having to 
> re-read height_field and image_map files (non-changing raster files) on 
> every frame render.

well, it's what you get when a software originally intended to
render single images is extended by an animation feature which
basically renders a lot of single images sequentially. It doesn't
hurt most users, as parse time is usually short compared to the
render time (some people here render single images which take
more than a day just by themselves).

I suppose your best bet is to compile a custom version of POV-Ray.
It won't need to support generic persistent variables, just a hack
specific to your animation to use a pointer to some preloaded image
data for just that one height_field and its texture.

BTW, did you try how fast the animation renders when you subsample
the images to 50% width/height (i.e. 25% image size)? If parse time
reduces by much more than a factor of 4 then your main problem is
not having enough memory.

Post a reply to this message

Brent Fraser wrote:
>   I'm new to Pov-Ray so I'm still trying to get up to speed on the design
>   philosophy.  My understanding so far: the Pov files describe a scene
>   containing objects with properties and a viewpoint (camera).  Some
>   properties change over time, driven by Pov's CLOCK variable.  Since the
>   properties are complex (can interact, etc) the scene needs to be
>   reconstructed (object properties re-calculated, possibly including the
>   camera postiion/orientation) for each frame (tick of the clock), and
>   this is done by re-parsing.  I get that; what I don't get is having to
>   re-read height_field and image_map files (non-changing raster files) on
>   every frame render.

Because POV-Ray has no idea if the images will change between frames. Or if
you use "image"+frame_number+".png". Absolutely no information is kept in
memory from one frame to another, except for the fairly-new clockless
animation feature.

Post a reply to this message

> -----Original Message-----
> From: Nicolas Alvarez [mailto:nic### [at] gmailcom]
> Because POV-Ray has no idea if the images will change between frames.
> Or if
> you use "image"+frame_number+".png". Absolutely no information is kept
> in
> memory from one frame to another, except for the fairly-new clockless
> animation feature.

Actually, it *should* be possible to use a global cache of image data.
When an image needs to be loaded during the parse stage, POV can check
whether or not the image is already in the cache.  If it is, then check
the timestamp on the physical file.  If the image is not in the cache,
or the timestamp is newer than the image in the cache (would happen if
you use the output of one frame when rendering the next), then load the
image from disk... otherwise, use the image in the cache.


...Ben Chambers
www.pacificwebguy.com

Post a reply to this message

Ben,

  I just finished implementing (hacking?) image file caching based only on the file
pathname, although I think your idea of using dates too is a good idea.  In my case
the height_fields and image_maps do not change during the rendering (if they did it
would be a data management failure!).

  On my single computer test it improved performance by about 40% (3 minutes per frame
instead of 5 minutes).  I expect if I reran the entire animation render (with 6
computers, 15 SDL/pov files, accessing by 400 mb of height_fields and image_maps on by
networked, slow RAID server), it would shave 40% to 80% off the render time.

Thanks!
Brent Fraser

Chambers wrote:
>> -----Original Message-----
>> From: Nicolas Alvarez [mailto:nic### [at] gmailcom]
>> Because POV-Ray has no idea if the images will change between frames.
>> Or if
>> you use "image"+frame_number+".png". Absolutely no information is kept
>> in
>> memory from one frame to another, except for the fairly-new clockless
>> animation feature.
> 
> Actually, it *should* be possible to use a global cache of image data.
> When an image needs to be loaded during the parse stage, POV can check
> whether or not the image is already in the cache.  If it is, then check
> the timestamp on the physical file.  If the image is not in the cache,
> or the timestamp is newer than the image in the cache (would happen if
> you use the output of one frame when rendering the next), then load the
> image from disk... otherwise, use the image in the cache.
> 
> 
> ...Ben Chambers
> www.pacificwebguy.com
> 
> 
>

Post a reply to this message