Distributing rendering in chunks is rather crude, it is far better to write 
code that spreads computational loops and recursion etc. over as many 
threads as possible and then run the code on a MOSIX cluster where the 
operating system automatically distributes the threads among the available 
CPUs.

I mentioned this recently as but got zero replies.
:o(

I wanted to know if anyone has profiled POVRay to see where all the work 
was being done so that I could use the Cilk system to create a massively 
threaded version of POVRay that could run across a MOSIX cluster.

There is also a PVM version of POV but last time I looked it was based on 
an old version. I was hoping to look at the 3.5 source code some time soon 
and use that as a basis for a Cilk-MOSIX_Povray variant.

This would have to be the Linux/Unix version as MOSIX is not an option for 
a closed source operating system such as Windows, unless Microsoft make a 
MOSIX version of Windows XP (XS?) :o)  available.


Sven Heinzel wrote:

> Nieminen Mika <war### [at] cctutfi> wrote:
> 
>> Another issue is that the radiosity code has to be modified in
>> order to be able to be calculated in parts.
> 
> Why is it not posible to render a picture with radiosity in parts? Does
> this mean +SRn and +ERn must not be used with radiosity scenes? I
> couldn't find any hint in the documentation.
> 
> Regards
> Sven

Post a reply to this message

In article <355### [at] 3-enet>, dan### [at] 3-enet says...
> Distributing rendering in chunks is rather crude, it is far better to write 
> code that spreads computational loops and recursion etc. over as many 
> threads

Why? I think, for performance, this is worse then rendering chunks, 
because the OS also needs CPU time for assigning tasks to each CPU.
And the more often the different threads/tasks change/interact, the more 
time it needs. So, for maximum performance it should IMHO be best to 
split the rendering in exactly as many different tasks as there are CPUs 
available, and then let each of them do their work completely on their 
own.

Lutz-Peter

Post a reply to this message

Also, we need a "binary" scene file, that the parsed scene can be stored in.
That way, the scene is parsed once, stored in a shared file (or the file can be
shipped to each rendering engine), then rendered by the farm.  Perhaps more
efficient than each render-farm element parsing the same scene in order to render
"chunks" of it.

Randy Hawley

Lutz-Peter Hooge wrote:

> In article <355### [at] 3-enet>, dan### [at] 3-enet says...
> > Distributing rendering in chunks is rather crude, it is far better to write
> > code that spreads computational loops and recursion etc. over as many
> > threads
>
> Why? I think, for performance, this is worse then rendering chunks,
> because the OS also needs CPU time for assigning tasks to each CPU.
> And the more often the different threads/tasks change/interact, the more
> time it needs. So, for maximum performance it should IMHO be best to
> split the rendering in exactly as many different tasks as there are CPUs
> available, and then let each of them do their work completely on their
> own.
>
> Lutz-Peter

Post a reply to this message

In article <355### [at] 3-enet> , Daniel Matthews <dan### [at] 3-enet>  
wrote:

> Distributing rendering in chunks is rather crude, it is far better to write
> code that spreads computational loops and recursion etc. over as many
> threads as possible and then run the code on a MOSIX cluster where the
> operating system automatically distributes the threads among the available
> CPUs.
>
> I mentioned this recently as but got zero replies.
> :o(

<http://www.students.tut.fi/~warp/povVFAQ/languageVFAQ.html#multiprocessor>


____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org

Post a reply to this message

In article <3BFDCF57.4026E69A@iquest.net> , Randy Hawley 
<rha### [at] iquestnet>  wrote:

> Also, we need a "binary" scene file, that the parsed scene can be stored in.
> That way, the scene is parsed once, stored in a shared file

It won't help increase the speed of parsing because most of the time is
spend on memory allocation (and processing #macros if you have them).

So there would be no fundamental improvement, but the implementation would
be rather time consuming and a lot of cross-platform issues would have to be
handled.  In conclusion, while it isn't impossible to modify POV-Ray to
output a "binary scene file" (it has been done before with 2.2), it doesn't
solve any problems, it just adds more problems.

    Thorsten

____________________________________________________
Thorsten Froehlich, Duisburg, Germany
e-mail: tho### [at] trfde

Visit POV-Ray on the web: http://mac.povray.org

Post a reply to this message

On Thu, 22 Nov 2001 23:23:52 -0500, Randy Hawley <rha### [at] iquestnet> wrote:

> Also, we need a "binary" scene file, that the parsed scene can be stored in.
> That way, the scene is parsed once, stored in a shared file (or the file can be
> shipped to each rendering engine), then rendered by the farm.

IMO it has sense only if the "binary" format supports constructions like
#macros, #while etc. Sometimes could be  more efficent to send short script with
#while(N<10000) and parse it on place instead of unrolled "binnary" list of
objects with slow bandwidth.

Also I wonder how accuracy problem will work when platforms on the farm will
support different accuracy.

ABX
--
#declare _=function(a,b,x){((a^2)+(b^2))^.5-x}#default {pigment{color rgb 1}}
union{plane{y,-3}plane{-x,-3}finish{reflection 1 ambient 0}}isosurface{ //ABX
function{_(x-2,y,1)|_((x+y)*.7,z,.1)|_((x+y+2)*.7,z,.1)|_(x/2+y*.8+1.5,z,.1)}
contained_by{box{<0,-3,-.1>,<3,0,.1>}}translate z*15finish{ambient 1}}//POV35

Post a reply to this message

Perhaps my statements would make more sense when I describe what I would be thinking
about putting together as my "rendering farm".

I own one "fast" computer (albeit not all that fast anymore), and several older, much
slower machines (all Wintel boxes right now - but I am considering dual-booting my
good one with Linux).  They are all networked on a 10Meg ethernet.

The fast computer may be able to parse some scenes about 100 times faster than the
slower ones.  It may be possible to transfer the memory constructs created by the
parsing process to the slower machines faster than they could generate them.  This
would be even more possible were I to have a 100 Meg ethernet.

So, I have a dream.  It may not be practical, but it is a dream.

The "accuracy problem" is supposed to be addressed by the POV program.  The team
mentions that it has worked to keep cross-platform problems at a minimum, including
random number generation.  I don't know enough about the differences in CPU
architecture to speak to differing accuracies in floating-point operations.

Randy Hawley



> On Thu, 22 Nov 2001 23:23:52 -0500, Randy Hawley <rha### [at] iquestnet> wrote:
>
> > Also, we need a "binary" scene file, that the parsed scene can be stored in.
> > That way, the scene is parsed once, stored in a shared file (or the file can be
> > shipped to each rendering engine), then rendered by the farm.
>
> IMO it has sense only if the "binary" format supports constructions like
> #macros, #while etc. Sometimes could be  more efficent to send short script with
> #while(N<10000) and parse it on place instead of unrolled "binnary" list of
> objects with slow bandwidth.
>
> Also I wonder how accuracy problem will work when platforms on the farm will
> support different accuracy.
>
> ABX
> --
> #declare _=function(a,b,x){((a^2)+(b^2))^.5-x}#default {pigment{color rgb 1}}
> union{plane{y,-3}plane{-x,-3}finish{reflection 1 ambient 0}}isosurface{ //ABX
> function{_(x-2,y,1)|_((x+y)*.7,z,.1)|_((x+y+2)*.7,z,.1)|_(x/2+y*.8+1.5,z,.1)}
> contained_by{box{<0,-3,-.1>,<3,0,.1>}}translate z*15finish{ambient 1}}//POV35

Post a reply to this message

Just a general comment:
The discussion of parallel rendering, a binary scene file format and/or
other radical scene description changes are suggested every few month.
However, everything has been said before and the decision to implement
either is not in the hands of those discussing it.  So no need for to people
waste their energy and time to discuss things they can neither change at the
moment nor will be responsible to change in the future...

    Thorsten

Post a reply to this message

Thorsten Froehlich wrote:

> Just a general comment:
> The discussion of parallel rendering, a binary scene file format and/or
> other radical scene description changes are suggested every few month.
> However, everything has been said before and the decision to implement
> either is not in the hands of those discussing it.  So no need for to people
> waste their energy and time to discuss things they can neither change at the
> moment nor will be responsible to change in the future...
> 
>     Thorsten
> 

"We know what is good for you".

Disclaimer: I really appreciate the hard work done by the pov team, a 
part of which includes the extremely long, painful and not rewarding 
process of hardening, packaging, and documenting the software we all use 
and enjoy.

Now, while I regret that these often sterile discussions always pop up 
from people who did not inquire about what has already been done and 
always want to reinvent the wheel (there are at least 5 different 
distributed versions after POVRay 3.1), this kind of statement sounds 
like an insult to me.

Following your often repeated statements, a good world would be a world 
without anybody implementing anything except under the official control 
of the One Team. A world without patches. A world without macros, 
isosurfaces, improved radiosity, light and vista buffers to quote a few, 
just because people discussing about them do not have the power to 
decide whether these features should be implemented or not.

If people want to try something, just let them do it. Even the current 
POVLEGAL allows them to. You won't believe me: it even encourages them 
to. Many of us are not professional programmers, thus the code resulting 
from our work is often bugged, poorly written, docummented and so on. 
But why should we not try to do things if:
- They do not exist
- We need them for some reason
- We have the will and the knowledge to implement them

Do you think that people who would think it nice to have a generic 
particle system, or something about cloth-like surfaces, or 
post-processing, or motion blur will just start thinking "this guy is 
right: we are not entitled to add our crap to the official work, let's 
stop our outrage"? I hope not. Call this proof of concept if you want, 
but let them discuss and try to implement and improve their ideas.

-- 

       __  __ __  __  _
|  | /  \  /  / |_  /  |/
\/\/ \__/ /_ /_ |__ \_ |\

Post a reply to this message

: If people want to try something, just let them do it.

  Did he say that people shouldn't try? I didn't see him saying that.
I think that he was talking about _discussions_ about the issue (if I
understood correctly, discussions of type "someone should do this and
this").

-- 
#macro N(D,I)#if(I<6)cylinder{M()#local D[I]=div(D[I],104);M().5,2pigment{
rgb M()}}N(D,(D[I]>99?I:I+1))#end#end#macro M()<mod(D[I],13)-6,mod(div(D[I
],13),8)-3,10>#end blob{N(array[6]{11117333955,
7382340,3358,3900569407,970,4254934330},0)}//                     - Warp -

Post a reply to this message