Regarding a starfield query in p.b.anim I noticed a peculiar
effect when using crackle at extreme scalings. The following
scene parses and renders in about a second at 640x480 (even
when just using a single thread) but then hangs for about 20
seconds before returning to idle mode. With scale .02 this
doesn't happen.

sky_sphere
{
   pigment
   {
     crackle
     form <1,1,0>
     scale .002
     color_map
     {
       [0 color rgb 100000]
       [1 color rgb 0]
     }
   }
}

I suspect the reason has to do with crackle cache hits dropping
from 99% to 20% so it probably bloats some internal data structure,
but it seems strange that freeing that would take so much longer
than building it in the first place. Maybe a case of O(n*n)? ;)

Tested on Win 7 64-bit with 3.7 beta 39 on an i7-970.

Post a reply to this message

Le 19/10/2010 11:39, Christian Froeschlin nous fit lire :

> I suspect the reason has to do with crackle cache hits dropping
> from 99% to 20% so it probably bloats some internal data structure,
> but it seems strange that freeing that would take so much longer
> than building it in the first place. Maybe a case of O(n*n)? ;)
> 
> Tested on Win 7 64-bit with 3.7 beta 39 on an i7-970.

Cannot reproduce on Linux 64-bits with 3.7 beta 39+@(#5167) on i7 too.
Even with scale 0.002 and 6400x4800 with AA 0.004, just a slow down to
compress the file.

Are you sure your MS-Win is fresh enough (swap/gui handle/what else) ?
Enough free mem ?

Post a reply to this message

Le_Forgeron wrote:

> Cannot reproduce on Linux 64-bits with 3.7 beta 39+@(#5167) on i7 too.
> Even with scale 0.002 and 6400x4800 with AA 0.004, just a slow down to
> compress the file.

There is no real drain on resources when rendering and plenty
of free memory. I wonder if it's specific to the windows version,
possibly OS memory allocation is less efficient when allocating
gazillions of tiny blocks there?

I now watched the behavior in the task manager:

Before rendering: 40 MB

After rendering: 200 MB (after about 1 second)

While "hanging": Memory usage slowly decreases but
at ever-increasing speed, when reaching 40MB after
about 20 seconds GUI goes "Ping".

Post a reply to this message

Le 19/10/2010 23:28, Christian Froeschlin a écrit :
> Le_Forgeron wrote:
> 
>> Cannot reproduce on Linux 64-bits with 3.7 beta 39+@(#5167) on i7 too.
>> Even with scale 0.002 and 6400x4800 with AA 0.004, just a slow down to
>> compress the file.
> 
> There is no real drain on resources when rendering and plenty
> of free memory. I wonder if it's specific to the windows version,
> possibly OS memory allocation is less efficient when allocating
> gazillions of tiny blocks there?
> 
> I now watched the behavior in the task manager:
> 
> Before rendering: 40 MB
> 
> After rendering: 200 MB (after about 1 second)
> 
> While "hanging": Memory usage slowly decreases but
> at ever-increasing speed, when reaching 40MB after
> about 20 seconds GUI goes "Ping".


I saw such pattern on linux with a nearly empty scene and a number of
thread about 1 less than core: each block is rendered so fast that each
data structure (rendered block) queues. It's what happen when a lot of
producers are faster than the single consumer for the rendered block
queue. Usually, the rendering is slow enough to not queue at all (except
one at a time). On gnome system monitor, it make a nice raising curve as
render is progressing, and once finished, the core usage drops far
before the memory begins to return to a lower state. If the picture size
is big enough and the file format use some compression, the writing of
the file can extend past the release of the extra block: the level is
nearly as when the rendering start (after parsing)). If no file output,
or no compression, usually everything drop back to empty system at the
same time (or very quick anyway)

But that can account only for a number of bytes about the picture size.
At 640 x 480, with 4 double per pixel (I'm lazy, I just do not check the
actual code now), it would explain only 9.6 MB
(and from the start of the rendering, you would have a basic footprint
of 5 double per pixels, about 12 MB for 640x480)

You might be right about the OS & C++ allocator: maybe they have some
smart garbage collector... and it's just failing in that case. Reusing
memory as soon as possible as its drawback too: it's harder to detect
illegal access to released objects, and it might provide fragmentation.

How does it perform with the benchmark scene ? (not the --benchmark or
/benchmark, or the menu entry; but from the samples scenes (benchmark is
also there)) at the same resolution (640 x 480).
Does it grow as much or less or bigger while rendering ?
Does the number of threads (-WT, from 2 to ...) have any impact ?
(try at 2, and at (number of core-1))

-- 
A good Manager will take you
through the forest, no mater what.
A Leader will take time to climb on a
Tree and say 'This is the wrong forest'.

Post a reply to this message