I have beta 35a running on an i7 and an AMD Phenom system. I've been adopting
radiosity for my scenes, and using the beta to speed things up with the two quad
core machines. There haven't been any problems until recently but my latest
scene froze the computer. I rebooted and launched the task manager window to
analyze CPU and memory utilization. I began with the i7 system with 6GB. The
render began using about 4 GB. After the radiosity pre-trace (the funny square
phase? :)) the memory allocation began to slowly increase at the rate of about
1GB per 10 seconds. This continued until the memory was full, then Windows
(arghh!) attempted to run in virtual memory (I assume because console I/O was
slow and disk was constanly busy).

I ran to the store and purchased another 6GB of RAM, installed it and tried
again with a total of 12GB. The same thing happened (only I was able to halt the
process in time when 11.8GB was reached).

Another strange thing... I noticed that there are many times when CPU
utilization is alarmingly low. When I run version 3.6 I seen 100% utilization of
one core. With 3.7b-35a it is bursty, and spends a lot of time running at 20%
utilization. During those times, the load seems to be fairly evenly divided
between the 4 cores.

The scene in question isn't anything special, just a landscape with a few
isosurfaces and planes. The isosurfaces do contain a lot of functions, which
create a lot of divergent surface details. To be perfectly clear, when I run the
same code without radiosity everything is fine. Also I ran the same code on
version 3.6 with radiosity turned on and everything is fine.

Ideas?

Post a reply to this message

On 12.02.10 17:31, Ken Willmott wrote:
> I ran to the store and purchased another 6GB of RAM, installed it and tried
> again with a total of 12GB. The same thing happened (only I was able to halt the
> process in time when 11.8GB was reached).
>
> Another strange thing... I noticed that there are many times when CPU
> utilization is alarmingly low. When I run version 3.6 I seen 100% utilization of
> one core. With 3.7b-35a it is bursty, and spends a lot of time running at 20%
> utilization. During those times, the load seems to be fairly evenly divided
> between the 4 cores.
>
> The scene in question isn't anything special, just a landscape with a few
> isosurfaces and planes. The isosurfaces do contain a lot of functions, which
> create a lot of divergent surface details. To be perfectly clear, when I run the
> same code without radiosity everything is fine. Also I ran the same code on
> version 3.6 with radiosity turned on and everything is fine.
>
> Ideas?

Radiosity, an experimental feature, changed from 3.6 to 3.7, so you may need 
to adjust your parameters. And it is, sadly, not uncommon for Windows to 
crash when it runs out of memory.

You also did not mention that you are running a 64-bit Windows and the 
64-bit version of POV-Ray. If you are not running both as 64-bit versions, 
POV-Ray will never get more than 3 GB of memory, regardless of the amount of 
RAM you install. This would certainly explain your low CPU usage, as it 
implies your system is spending most of its time with swapping.

	Thorsten

Post a reply to this message

"Ken Willmott" <ken### [at] rogerscom> wrote:

> Another strange thing... I noticed that there are many times when CPU
> utilization is alarmingly low. When I run version 3.6 I seen 100% utilization of
> one core. With 3.7b-35a it is bursty, and spends a lot of time running at 20%
> utilization. During those times, the load seems to be fairly evenly divided
> between the 4 cores.

If it is doing a lot of swapping, then the CPU usage may drop during swaps.

>
> The scene in question isn't anything special, just a landscape with a few
> isosurfaces and planes. The isosurfaces do contain a lot of functions, which
> create a lot of divergent surface details. To be perfectly clear, when I run the
> same code without radiosity everything is fine. Also I ran the same code on
> version 3.6 with radiosity turned on and everything is fine.
>
> Ideas?

I am really curious about how your radiosity settings are.  Also, what is the
peak memory usage without radiosity?

-Reactor

Post a reply to this message

Thanks for the comments... there are good suggestions there. I prepared a test
file so that any of you could try it out. I verified that both OS and beta are
running 64 bits. I discovered that there are problems even with radiosity
commented out. Namely, CPU usage is not 100%, and control does not come back to
the GUI after the render is complete. With radiosity (using default values)
enabled, the task levels off at 7.89GB. I think it is only because I simplified
the scene. Many minutes elapse before it releases the memory, and returns
control (or I lose patience and kill the app). Here is the source code:

#include "functions.inc"
#include "colors.inc"


global_settings
{
        ambient_light 0.001
        radiosity
        {
        }
}

#declare valley_radius = 8.0;

camera {
        location <valley_radius*100, -45, 0>
        up <0, 1,  0>
        right x*1600/1200
        angle 60
        look_at <valley_radius*100,-100,4000>
}

light_source
{
        < -60, 30, -30>
        color rgb <255/256, 229/256, 123/256>
        parallel
}

fog {fog_type 1 distance 2000 color Gray}

#declare P1=function
                {pattern{wrinkles turbulence 0.3}}
#declare P2=function
                {pattern{crackle turbulence 0.3}}
#declare P3=function
                {pattern{bozo turbulence 0.3}}

#declare ROCK_TEXTURE_SCALE = 0.5;
#declare RockTexture = function
                {
                P1(x,0,z)*1/4*ROCK_TEXTURE_SCALE
                +P2(x,0,z)*3/4*ROCK_TEXTURE_SCALE
                }

#declare RockTexture2 = function
                {
                P3(x*2,0,z*2)*1/4*ROCK_TEXTURE_SCALE
                +P3(x,0,z)*3/4*ROCK_TEXTURE_SCALE
                }

#declare  F_valley_sides = function {exp(-(2-2*abs(x))*(2-2*abs(x)))}
#declare  F_valley_contour = function {exp(-(x*x*x*x/4))}
#declare  F_river_contour = function {exp(-(x*x*x*x*4096))}

#declare ValleySurface =
function
{
        y
        + F_valley_contour(sqrt(x*x+z*z)-valley_radius, y, 0) //SOR of valley
        - F_valley_sides(sqrt(x*x+z*z)-valley_radius, y, 0)*RockTexture(x,0,z)*2
//SOR of valley sides
        + F_river_contour
                        (
                                sqrt(x*x+z*z)
                                -valley_radius
                                +(0.5-f_bozo(x*2, 0, z*2))/2,
                                y,
                                0
                        )/8 //SOR of river w/ meander
        - F_river_contour
                        (
                                sqrt(x*x+z*z)
                                -valley_radius
                                +(0.5-f_bozo(x*2, 0, z*2))/2,
                                y,
                                0
                        )*RockTexture(x*64,0,z*64)/8 //SOR of river
        - RockTexture2(x*8,0,z*8)*0.02
}

//Valley
isosurface
{
        function
        {
        ValleySurface(x,y,z)
        }
        max_gradient 8
        contained_by{box{<-80,-10,-20>,<80,3,20>}}
        texture
        {
                pigment {colour rgb <171/256, 23/256, 15/256>}
                finish {specular 0.5  diffuse 0.5}
        }

        scale <100, 50, 100>
}

// some water...
plane
{
        y, -51
        hollow on
        pigment { rgbt <0.1, 0.6, 0.5, 0.8> }
        normal
        {
                bozo 1.6
                scale <0.5, 1.0, 0.5>
        }
        finish
        {
                ambient 0.05
                diffuse 0.2
                reflection 0.1
        }
        interior
        {
                ior 1.3
                media { absorption <1.8, 1.0, 0.8>/2 }
        }
}

Post a reply to this message

> Thanks for the comments... there are good suggestions there. I prepared a test
> file so that any of you could try it out. I verified that both OS and beta are
> running 64 bits. I discovered that there are problems even with radiosity
> commented out. Namely, CPU usage is not 100%, and control does not come back to
> the GUI after the render is complete. With radiosity (using default values)
> enabled, the task levels off at 7.89GB. I think it is only because I simplified
> the scene. Many minutes elapse before it releases the memory, and returns
> control (or I lose patience and kill the app). Here is the source code:
>

The use of the crackle pattern, in any way, result in more memory load. 
Anyway, it should not acount for the load you are seeing.

It won't affect your memory problem, but will affect the speed of the 
isosurfaces:
Change:
#declare  F_valley_contour = function {exp(-(x*x*x*x/4))}
#declare  F_river_contour = function {exp(-(x*x*x*x*4096))}

to:
#declare  F_valley_contour = function {exp(-(pow(x,4)/4))}
#declare  F_river_contour = function {exp(-(pow(x,4)*4096))}

Every time you encounter a variable, even the same used several times in 
a row, you need to do a search in the variable table. It does take more 
time than to elevate to a power.

Whenever you have x*x use pow(x,2).
Do the same for y*y and z*z.

For the radiosity, it's beter to NOT use ambient_lights in the 
global_settings, but to use #default{finish{ambient 0}}


Alain

Post a reply to this message

On 14.02.10 03:30, Ken Willmott wrote:
> Thanks for the comments... there are good suggestions there. I prepared a test
> file so that any of you could try it out. I verified that both OS and beta are
> running 64 bits. I discovered that there are problems even with radiosity
> commented out. Namely, CPU usage is not 100%, and control does not come back to
> the GUI after the render is complete.

The low CPU usage is likely related to the use of crackle, which requires 
some data to be shared between threads. This in turn creates dependencies 
between render threads. The performance of multi-threaded ray tracing 
exclusively depends on the independence of render threads.

Since crackle also happens to be one of the less suitable functions for 
integration performed to render it is an isosurface, it takes very long time 
to find intersections with the isosurface involving a real lot of crackle 
pattern evaluations. The POV-Ray statistics likely show this.

Freeing memory in this state, for various reasons, may also take quite some 
time. Again, the statistics may provide some indication on what is going on.

	Thorsten, POV-Team

Post a reply to this message

When I removed the crackle function from my main project, all the problems
disappeared. Memory remained stable, slightly less than 2.0GB. It will be a
painful loss, since it made such beautiful rock surfaces. I guess I will end up
using something like bozo+wrinkles instead.

It is my impression that virtual memory is useful mostly to make memory
management easier, rather than to increase the working memory capacity. It can't
work with 100% physical memory allocated because of excessive disk swapping. I
think this is a fundamental constraint that is common to all OS, whether it is
Windows or Linux or whatever.

So, in this case, the issue is only that what should have only been a painfully
slow render in physical memory, which some users might have found an acceptable
price to pay for a certain effect, became instead a full-blown failure of the
OS. If there is some way in that case, to limit the expansion of memory or exit
with an error, it would probably be better.

I also realize that with beta versions, the development issues are diverse and
priority is correctly given to primary design features above error and UI
issues.

Thanks, everyone for the help! I can now continue with my project, and I have
other radiosity questions which I will move to the other newsgroup.


Thorsten Froehlich <tho### [at] trfde> wrote:
> On 14.02.10 03:30, Ken Willmott wrote:
> > Thanks for the comments... there are good suggestions there. I prepared a test
> > file so that any of you could try it out. I verified that both OS and beta are
> > running 64 bits. I discovered that there are problems even with radiosity
> > commented out. Namely, CPU usage is not 100%, and control does not come back to
> > the GUI after the render is complete.
>
> The low CPU usage is likely related to the use of crackle, which requires
> some data to be shared between threads. This in turn creates dependencies
> between render threads. The performance of multi-threaded ray tracing
> exclusively depends on the independence of render threads.
>
> Since crackle also happens to be one of the less suitable functions for
> integration performed to render it is an isosurface, it takes very long time
> to find intersections with the isosurface involving a real lot of crackle
> pattern evaluations. The POV-Ray statistics likely show this.
>
> Freeing memory in this state, for various reasons, may also take quite some
> time. Again, the statistics may provide some indication on what is going on.
>
>  Thorsten, POV-Team

Post a reply to this message

"Ken Willmott" <ken### [at] rogerscom> wrote:
> When I removed the crackle function from my main project, all the problems
> disappeared. Memory remained stable, slightly less than 2.0GB. It will be a
> painful loss, since it made such beautiful rock surfaces. I guess I will end up
> using something like bozo+wrinkles instead.
>
> It is my impression that virtual memory is useful mostly to make memory
> management easier, rather than to increase the working memory capacity. It can't
> work with 100% physical memory allocated because of excessive disk swapping. I
> think this is a fundamental constraint that is common to all OS, whether it is
> Windows or Linux or whatever.


You mentioned having quite a bit of RAM.  Have you tried turning virtual memory
off and using a RAM disk to see how it does?  RAM disks are not difficult to set
up, and it may be worth the effort.

-Reactor

Post a reply to this message

On 14.02.10 20:47, Reactor wrote:
> You mentioned having quite a bit of RAM.  Have you tried turning virtual memory
> off and using a RAM disk to see how it does?

Given that turning virtual memory off is impossible in Windows that seems 
like an odd suggestion...

	Thorsten

Post a reply to this message

Thorsten Froehlich wrote:
> Given that turning virtual memory off is impossible in Windows that 
> seems like an odd suggestion...

Turning virtual memory off in Windows is trivial. I don't think you can turn 
off virtual addressing, but that shouldn't have an impact on this problem.

Control panel->System->Advanced->Performance->Advanced->No page file

Admittedly, it's behind two "advanced" tabs, but the help tells you how to 
get there too if you put "virtual memory" into the search bar of the control 
panel.

-- 
Darren New, San Diego CA, USA (PST)
   Forget "focus follows mouse." When do
   I get "focus follows gaze"?

Post a reply to this message