With some of the recent posts on environmental HDRI image use, I played 
today in the povr branch with .hdr and .exr 4k x 2k images. First, 
viewing them via rtr/kla and then playing a little with potential HDRI 
edits(2).

The frame rates are surprisingly good even on my little i3 rotating HDRI 
images full circle. In fact for small windows it's quick - for a while...

It looks like large amounts of memory are allocated to back the large 
HDRI images and that memory (or a substantial part of it) is freed at 
the end of each 'frame.'

The free's cannot keep up after a certain fps rate(1) and the memory 
attached to the povray process quickly jumps to many gigabytes. If 
lucky, you reach an equilibrium before you start to page (or crash).

For now, the frame rates can be slowed with strong AA settings. For the 
4k x 2k HDRIs with which I was playing, I had to use AA costly enough to 
drop to around 2 fps to keep the memory continuously under 500MB.

Note: There are github pull requests and issues related to not freeing 
image memory during animations which might be of use - should one of us 
find a 'todo' to pop the memory bubble issue.

Aside: When the frame rate gets high, closing the preview window itself 
works much better than 'q' (quit) or 'p' (pause). Or, on linux/unix, you 
can have a kill command ready to go on the povray process id should the 
memory consumption start to get out of hand...

Bill P.

(1) - I suspect the slow free might be due the use of c++11 vectors - 
but that's a guess. For rtr/kla it would, of course, be better not to 
allocate and free the memory, but rather re-use it.

(2) - The idea being to modify HDRIs with an rtr/kla 360 preview. After 
which, a render using same scene, but a single spherical camera, would 
be used to generate an updated HDRI image.

Post a reply to this message

William F Pokorny <ano### [at] anonymousorg> wrote:
> With some of the recent posts on environmental HDRI image use, I played
> today in the povr branch with .hdr and .exr 4k x 2k images. First,
> viewing them via rtr/kla...
>
> The frame rates are surprisingly good even on my little i3 rotating HDRI
> images full circle. In fact for small windows it's quick - for a while...
>
> It looks like large amounts of memory are allocated to back the large
> HDRI images and that memory (or a substantial part of it) is freed at
> the end of each 'frame.'
>

[in Windows]
Confirmed in v3.8.0 beta 1 (v3.7.0 can't use this feature, otherwise I would
have tested it there too.)

While trying to track down where the memory increase might be coming from, I
instead wrote a *bare* test scene-- with nothing but the changing cameras! Yet
the memory-use still increases.

Try running just this scene in your povr branch to see what happens:
 -----
#version 3.8;
global_settings{assumed_gamma 1.0}

// MULTIPLE CAMERAS:
#declare CK = 0;

#while(CK <= 1.0)
camera {
  perspective
  location  <0, 25, 40>
  look_at   <0,0,0>
  right     x*image_width/image_height
  angle 67 // 67
  rotate 360*CK*y
}
#declare CK = CK + .05;
#end
 -----

[But WARNING: On my i7 8-core machine, this runs SO fast that the 'stop' button
doesn't reliably work; I had to shut down POV-ray instead. OR, what also works
for me: Hit 'pause'-- which does pause the looping animation temporarily-- then
hit 'stop'. The looping still proceeds x-number of times before coming to a
final halt-- but the memory use starts to slowly *decrease* while that happens.
That's interesting by itself.]

My own 'working hypothesis' at the moment is that ALL of a scene's code and
elements are being duplicated for each 'new' loop, not just any image_maps...
and that each loop's 'new' memory use adds to the previous total.

Post a reply to this message

On 10/5/21 5:42 PM, Kenneth wrote:
> [in Windows]
> Confirmed in v3.8.0 beta 1 (v3.7.0 can't use this feature, otherwise I would
> have tested it there too.)
> 
> While trying to track down where the memory increase might be coming from, I
> instead wrote a*bare*  test scene-- with nothing but the changing cameras! Yet
> the memory-use still increases.
> 
> Try running just this scene in your povr branch to see what happens:
>   -----
> #version 3.8;
> global_settings{assumed_gamma 1.0}
> 
> // MULTIPLE CAMERAS:
> #declare CK = 0;
> 
> #while(CK <= 1.0)
> camera {
>    perspective
>    location  <0, 25, 40>
>    look_at   <0,0,0>
>    right     x*image_width/image_height
>    angle 67 // 67
>    rotate 360*CK*y
> }
> #declare CK = CK + .05;
> #end
>   -----
> 
> [But WARNING: On my i7 8-core machine, this runs SO fast that the 'stop' button
> doesn't reliably work; I had to shut down POV-ray instead. OR, what also works
> for me: Hit 'pause'-- which does pause the looping animation temporarily-- then
> hit 'stop'. The looping still proceeds x-number of times before coming to a
> final halt-- but the memory use starts to slowly*decrease*  while that happens.
> That's interesting by itself.]
> 
> My own 'working hypothesis' at the moment is that ALL of a scene's code and
> elements are being duplicated for each 'new' loop, not just any image_maps...
> and that each loop's 'new' memory use adds to the previous total.

Thanks! I didn't think to try larger render windows with just cameras. 
My non HDRI stuff was what I already had - small isosurface changing / 
morphing things and there memory is always small, but the frame rate and 
render sizes are smallish too.

If you get a chance, I'd be interested in a single thread run on your 
machine/windows. I'm actually getting better reported FPS that way than 
using 2 cores / 4 threads - which on the surface doesn't make much sense.

Playing around just now I had the thought to try -D to turn off the 
preview display and it does change the rate at which the memory bubble 
grows.

Aside 1: You've probably noticed too that pause/un-pause screws up the 
fps reporting.

Aside 2: Given what we see here, I'm wondering how much the freeing and 
re-allocation of memory is costing us 'during' normal single frame 
renders. c++17 / c++20 I understand will have more in the way of inbuilt 
memory pre-allocation / re-allocation (for memory locality of reference) 
and control - not that I currently have a clue how to make use of any of 
those features at the moment... :-)

Aside 3: I run rtr/kla renders with -cc and -fn too, to be sure I'm not 
creating the continuation and image files - or portions thereof. Which 
likely also changes behavior, but not explored much.

Bill P.

Post a reply to this message

William F Pokorny <ano### [at] anonymousorg> wrote:
> On 10/5/21 5:42 PM, Kenneth wrote:
> >
> > My own 'working hypothesis' at the moment is that ALL of a scene's code and
> > elements are being duplicated for each 'new' loop, not just any
> > image_maps... and that each loop's 'new' memory use adds to the previous
> > total.
>
> If you get a chance, I'd be interested in a single thread run on your
> machine/windows. I'm actually getting better reported FPS that way than
> using 2 cores / 4 threads - which on the surface doesn't make much sense.
>

I ran a series of thread tests, with the preview display OFF, and got some
interesting and mysterious results.

Running my bare 'cameras only' test scene, and letting the kla/rtr animation run
and loop continuously for exactly one minute:

1A) With:
preview display OFF
1 thread-- Work_Threads=1 in my .ini file
Result: NO increase in used memory(!), although the value dances around a bit as
expected (just a result of the memory being cleared after each loop, I suppose).
About 35MB average
20.5 fps, consistent

1B)
Same as A), 1 thread, but with the preview display ON (800X600 render):
The memory use *increases* to 2050 MB by the 1 minute mark.
16.6 fps, consistent more or less

The 1A) combination of 1 thread and NO display preview is the only one of my
tests where the memory did not increase. Here, the preview display seems to be
the root cause of the increasing memory.

However...
2) Using 2 threads, with preview display still OFF:
Memory use increases to 2040 MB anyway
30.3fps-- but *very slowly* starts climbing, by hundredths of a second

....and every additional thread that's used causes a FASTER memory increase. With
4 threads, it climbed to 11,723 MB! That's as far as I dared to go. So in these
>1-thread test cases, it looks like multi-threading itself is the culprit,
somehow.

(BTW, file output being on or off does not affect any of these results, as far
as I can tell.)

So maybe kla/rtr was never meant to be run in a multi-threaded environment?

-----
BTW: I need to modify my previous comments about the trouble with the 'stop'
button, at least in Windows. It actually *does* work-- but certainly not
immediately *when using more than 1 thread*. In such cases, the looping
animation continues to run for awhile, but only while POV-ray slowly RELEASES
its resulting big memory load. Once the memory value falls to POV-ray's 'idle'
value, the looping animation finally stops. None of this happens when using only
*1* thread, though; the 'stop' button works instantly.

Post a reply to this message

"Kenneth" <kdw### [at] gmailcom> wrote:

>
> I ran a series of thread tests, with the preview display OFF, and got some
> interesting and mysterious results.
>

Well, I'm getting some screwy results now. Just when I thought that my previous
tests were 'definitive' (more or less), some *new* and mysterious behavior
arises.

I just ran one of my 'typical' older (v3.8) scenes that I had already set up for
kla/rtr, with an almost identical #while-loop camera like my 'bare' test, and
with image_maps, objects, lights, etc. And I used all 8 cores/16 threads of my
machine, AND turned the preview display ON...

....And that scene runs with NO increase in memory, loop after loop! I don't get
it. I feel like I'm comparing bananas to footballs, with no logical basis for my
tests. :-<  If anything, *that* scene should trigger the increase in memory, not
my 'bare' scene.

Here's *another* screwy thing (which may be Windows-specific?):
Assume that I have turned both of these off:
preview display OFF
output file OFF

In POV-ray's GUI, I have to choose *some* particular render resolution (which
gets chosen from my 'quickres.ini' file) to run a scene-- even though there is
now NO image output and NO preview. But I discovered that the res I pick *does*
affect the fps speed of the looping animation. Running my 'bare' test scene with
8 cores/16 threads, this is what happens:
At 160X120 res:   90fps (!)
At 1280X960 res:  9.5fps

I didn't realize this earlier. And I have not yet determined if this ghostly
change-of-res (AND fps rate) affects how the memory use increases...or does not
increase.

Madness!

Post a reply to this message

On 10/7/21 4:09 AM, Kenneth wrote:
> "Kenneth" <kdw### [at] gmailcom> wrote:
> 
>>
>> I ran a series of thread tests, with the preview display OFF, and got some
>> interesting and mysterious results.
>>
> 
> Well, I'm getting some screwy results now. Just when I thought that my previous
> tests were 'definitive' (more or less), some *new* and mysterious behavior
> arises.
> 
> I just ran one of my 'typical' older (v3.8) scenes that I had already set up for
> kla/rtr, with an almost identical #while-loop camera like my 'bare' test, and
> with image_maps, objects, lights, etc. And I used all 8 cores/16 threads of my
> machine, AND turned the preview display ON...
> 
> ....And that scene runs with NO increase in memory, loop after loop! I don't get
> it. I feel like I'm comparing bananas to footballs, with no logical basis for my
> tests. :-<  If anything, *that* scene should trigger the increase in memory, not
> my 'bare' scene.
> 
> Here's *another* screwy thing (which may be Windows-specific?):
> Assume that I have turned both of these off:
> preview display OFF
> output file OFF
> 
> In POV-ray's GUI, I have to choose *some* particular render resolution (which
> gets chosen from my 'quickres.ini' file) to run a scene-- even though there is
> now NO image output and NO preview. But I discovered that the res I pick *does*
> affect the fps speed of the looping animation. Running my 'bare' test scene with
> 8 cores/16 threads, this is what happens:
> At 160X120 res:   90fps (!)
> At 1280X960 res:  9.5fps
> 
> I didn't realize this earlier. And I have not yet determined if this ghostly
> change-of-res (AND fps rate) affects how the memory use increases...or does not
> increase.
> 
> Madness!
> 
Hmm, Alright. Thank you for the data. I'm seeing odd results too. There 
are reasons - and likely a mix on them. We just don't know at the moment 
what the reasons are!

The only sure thing I see is slowing down the frame rate helps with the 
memory bubble. I've got and itch / vague recollection of having run 
across code comments about some memory handling trade off with rtr/kla.

Aside: I believe the entire reason the real time / continuous render 
options were created was to show off multiple core / hyper threaded 
CPUs. IIRC Thorsten and Chris did it in a hurry and it's more or less 
been in that demo state since.

FYI - I'm busy with real life until early to mid next week.

Bill P.

Post a reply to this message

William F Pokorny <ano### [at] anonymousorg> wrote:
> On 10/7/21 4:09 AM, Kenneth wrote:

> > Assume that I have turned both of these off:
> > preview display OFF
> > output file OFF
> >
> > In POV-ray's GUI, I have to choose *some* particular render resolution
> > (which gets chosen from my 'quickres.ini' file) to run a scene-- even
> > though there is now NO image output and NO preview. But I discovered
> > that the res I pick *does* affect the fps speed of the looping animation.
> > Running my 'bare' test scene with
> > 8 cores/16 threads, this is what happens:
> > At 160X120 res:   90fps (!)
> > At 1280X960 res:  9.5fps
> >

Oops; I was getting tired and brain-drained last night. Those stats are for my
'typical' scene. The 'bare' cameras-only scene produced these:
At 160X120 res: 900fps (!!!)
At 1280X960 res: 9.9fps
>
> The only sure thing I see is slowing down the frame rate helps with the
> memory bubble.

I was beginning to see that result as well, but could not definitely say yea or
nay.

> Aside: I believe the entire reason the real time / continuous render
> options were created was to show off multiple core / hyper threaded
> CPUs. IIRC Thorsten and Chris did it in a hurry and it's more or less
> been in that demo state since.

And it's still a great feature that I love to use.

Post a reply to this message