Hello,

This is my first post here. I have been playing with the idea of doing a 
render farm. My current thinking is to use cgi (perl) and a central server 
that keeps track of some basic state information. The POV scene uses this 
state information to render it's animation. So far I have two state 
schema's. One is a simple TIME variable that is incremented on each request 
to the server. 

The second is a sequence/duration thing. The state is maintained in a .inc 
file that looks like this.

#declare sequence_array = array[2,3,2]
#declare sequence_current = 2;
#declare sequence_index = 2;
#declare sequence_status = 'complete';

Each call to the server increments the sequence_index. When a sequence_index 
> sequence_array[sequence_current] I reset the index to 0 then increment the 
sequence_current. What this basically comes down to is a "what time is it?" 
thing. 

I use will be using perl to call the server. The server will return the 
state.inc file (as text) and the render client will use write this to it's 
local copy of state.inc, then shell out to pov to render that frame.

There is an enlist phase in this where a render client get's information 
about the project which includes things like:
> Where to ftp the result file.
> The project file list.
> other things the client would want to know.

 ---> What do YOU do? < ---
There are lots of folks out there coding POV scenes and lot's of ways to get 
the job done. It would be very helpful to me if you guys gave me some input 
on how you approach the state problem. For instance you might say "I use the 
interpolate function to determine where my objects vector is and use Clock 
to calculate where I am at any given time." In a case like this the master 
server might simply increment the Initial_Frame (which is like the first 
schema) I mentioned.
 ---> What do YOU do? < ---

Anyway..

I thought I would check around to see what kind of info I could find and 
found this link:

http://www.badcheese.com/~steve/index.php?page_title=Povray

Pretty cool. It delivers the product in a very simple way.

Many of the links I visited were defunct.. mia.. 404. I know about the linux  
binary distribution and have a linux box but I am interested in a cross 
platform thing.

Anyway, to get to the point. I have spent a lot of time looking here and 
thought I would post to see if anyone knows of any good render farm projects 
or links. Also, has anyone here approaced the problem themselves and if so, 
can you tell me a little about your project? Can you elaborate on the  
---> What do YOU do? < --- section?

Hope this is clear.

cheers.

Post a reply to this message

Do you plan to make a cross-platform thing able to render things shared over
the (inter)net ?

-- Theo
----------------------------------------------------------------------------
---------------------------------
Distributed Network-Rendering or Local SMP-Rendering on all CPU's you have.
With SMPOV und
POV-Ray 3.5. & NEW: Mega-POV 1.0 * Download free at:
http://www.it-berater.org/smpov.htm

Post a reply to this message

hi Ian!

Ian wrote:
> [...] I have been playing with the idea of doing a
> render farm. My current thinking is to use cgi (perl) and a central server 
> that keeps track of some basic state information. The POV scene uses this 
> state information to render it's animation. So far I have two state 
> schema's. One is a simple TIME variable that is incremented on each request 
> to the server. 
> The second is a sequence/duration thing. The state is maintained in a .inc 
> file that looks like this. 
 > [...]
> Anyway, to get to the point. I have spent a lot of time looking here and 
> thought I would post to see if anyone knows of any good render farm projects 
> or links. Also, has anyone here approaced the problem themselves and if so, 
> can you tell me a little about your project? Can you elaborate on the  
> ---> What do YOU do? < --- section?

i did something like that using perl, the "povray render server";
see http://www.psico.ch/index.php?page=prs .

it runs its own protocol on top of tcp/ip, works directly
with the supplied .ini file, parses it and builds a new one
for any client requesting a render job.  single frames
(from .pov files) are not split, however, as i basically
only used prs to render animations with a lot of frames.
then, the full set of files needed to render part of an
animation are sent to the client (including .pov, .inc and
image files), which in turn will return the rendered images
before requesting another job.

if you already know perl (i assume you do), then you should
have no trouble understanding the way it works by looking at
the code.  there also is a readme included in the zip archive
describing how to configure everything needed.

note though that i was developping it at a time when i had
no unix box at hand (and did hardly know perl before), so it
is only tested on perl for win32; but getting it to work on a
linux box would only require a few minor changes.

if that is what you'd like to do, and if i can provide further
help, please let me know.  actually, i was already thinking of
doing the port myself, because i've just got a new 1.3 GHz
celeron gateway running linux with over 95% cpu idle time... ;)

also, the code in the server's main loop that waits for clients'
requests and handles them, is *very* bad.  now that i have
learned more, i'd do it with a simple state machine and fork()
the handlers off (which won't work with windoze-perl :/ )...

anyway, i recommend that you keep your state information as
simple as possible, and that you do not touch the time values
yourself.  it's much simpler to change the subset_start/end_-
frame numbers only, and let povray figure out the actual time
itself - except if you insist on changing the initial/
final_frame values as well; but this will make things *very*
(and unnecessarily) complicated!

OTOH, smpov is an alternative. i guess that would mean less
work! :D


i hope this helps.

g.

-- 
"Beware of bugs in the above code. I have only proved it
correct, not tried it."  -- Donald Knuth
++ mailto:gim### [at] psicoch ++ http://www.psico.ch/ ++

Post a reply to this message

> OTOH, smpov is an alternative. i guess that would mean less
> work! :D

Are you shure about that :-) ?

I am just mailing with Ian, cause the documentation of SMPOV is really a
mess,
so a user has many chances to make mistakes and it doesn't work.

I did SMPOV primarily for myself, so I did not write nowhere much how to use
it.
Doesn't mean its not needed. I just didn't do it. So I an tried it and had a
lot of trouble.

Just two things if anyone else has trouble with SMPOV:     ...
- Ian sent me a corrected text for SMPOV, it will be included in the next
version (thanks a lot Ian !)
- If you cannot render your file, open the com-folder and put all files
which belong to that current scene
  you want to render there. Remember: the source folder is not accessible to
the clients and
  the current SMPOV-Version only copies the ".pov"-file not any necessary
".inc" files or pictures.

-- Theo
----------------------------------------------------------------------------
-----------------------------
Distributed Network-Rendering or Local SMP-Rendering on all CPU's you have.
With SMPOV und
POV-Ray 3.5. & NEW: Mega-POV 1.0 * Download free at:
http://www.it-berater.org/smpov.htm >

Post a reply to this message

Ian <pov### [at] nebulouscorecom> wrote in
news:Xns### [at] 204213191226: 

> Hello,
> 
> This is my first post here. I have been playing with the idea of doing
> a render farm. 

There are several existing projects that I know of out there.  A summary 
of them is at the end of this reply.  I would suggest starting simple. Do 
it frame by frame.  Anything else gets really complicated, really buggy, 
really fast.  Based on the work I have done for the Internet Movie 
Project I have boiled the process down to this:

The server just needs to do three things.

    1) Issue frames
    3) Track frames( Preferably in a database)
    2) Receive frames

The client just needs to do three things
    1) Get a frame
    2) Render a frame
    3) Return a frame

Our design goals are:

    * Cross platform client/server
    * http communication 
    * web based management

Obviously the devil is in the details.  Both Gimli and IMP have made the 
source available if you want to see what has been done.  The server 
portion of IMPFarm is still two ASP pages.  They are being converted to 
perl.  If you want to join the IMPFarm project just sign up. You can see 
the work we have done so far if you look at

   http://test.imp.org/impfarm/


Tom




POV-Ray RenderFarms
=======================================
IMPFarm
    Language=Perl
    Renderer=POV-Ray
    http://sourceforge.net/projects/impfarm/

Povray Render Server(Gimli already replied to this thread)
     Language=Perl
     Renderer=POV-Ray
     http://www.psico.ch/index.php?page=prs

The PVM patch for POV-Ray
     Language = C++
     Renderer = POV-Ray
     http://pvmpov.sourceforge.net/




Non POV-Ray projects
=======================================
BORG - BORG Open Rendering GUI
     Language = Java
     Renderer = Renderman compliant
     http://www.project-borg.org/home/index.html

RenderFarm.NET
     Language = XML
     Renderer = Maya
     http://www.renderfarm3d.com/

And these projects on source forge look dead:
     http://sourceforge.net/projects/renderfarm/
     http://sourceforge.net/projects/littlefarm/
     http://sourceforge.net/projects/barf/

Post a reply to this message

Tom Galvin wrote:
> Ian <pov### [at] nebulouscorecom> wrote in
> news:Xns### [at] 204213191226: 
> 

Nice projects. I really should delve into them more. I also have 
recently been thinking about this, and I thought that maybe it could be 
interesting also to look into clustering techniques like Mosix and 
OpenMosix. It would be nice if a frame could be divided. Yes, I am aware 
of several problems related to this (radiosity, etc.), but I thought 
that for instance current projects with Mosix like netram would be helpful.

I don't know how it could apply to PoV-ray (haven't really read the 
sources yet), but maybe someone knows more about this and can say 
something about it...

Just a thought.

-Nathan

> 
>>Hello,
>>
>>This is my first post here. I have been playing with the idea of doing
>>a render farm. 
> 
> 
> There are several existing projects that I know of out there.  A summary 
> of them is at the end of this reply.  I would suggest starting simple. Do 
> it frame by frame.  Anything else gets really complicated, really buggy, 
> really fast.  Based on the work I have done for the Internet Movie 
> Project I have boiled the process down to this:
> 
> The server just needs to do three things.
> 
>     1) Issue frames
>     3) Track frames( Preferably in a database)
>     2) Receive frames
> 
> The client just needs to do three things
>     1) Get a frame
>     2) Render a frame
>     3) Return a frame
> 
> Our design goals are:
> 
>     * Cross platform client/server
>     * http communication 
>     * web based management
> 
> Obviously the devil is in the details.  Both Gimli and IMP have made the 
> source available if you want to see what has been done.  The server 
> portion of IMPFarm is still two ASP pages.  They are being converted to 
> perl.  If you want to join the IMPFarm project just sign up. You can see 
> the work we have done so far if you look at
> 
>    http://test.imp.org/impfarm/
> 
> 
> Tom
> 
> 
> 
> 
> POV-Ray RenderFarms
> =======================================
> IMPFarm
>     Language=Perl
>     Renderer=POV-Ray
>     http://sourceforge.net/projects/impfarm/
> 
> Povray Render Server(Gimli already replied to this thread)
>      Language=Perl
>      Renderer=POV-Ray
>      http://www.psico.ch/index.php?page=prs
> 
> The PVM patch for POV-Ray
>      Language = C++
>      Renderer = POV-Ray
>      http://pvmpov.sourceforge.net/
> 
> 
> 
> 
> Non POV-Ray projects
> =======================================
> BORG - BORG Open Rendering GUI
>      Language = Java
>      Renderer = Renderman compliant
>      http://www.project-borg.org/home/index.html
> 
> RenderFarm.NET
>      Language = XML
>      Renderer = Maya
>      http://www.renderfarm3d.com/
> 
> And these projects on source forge look dead:
>      http://sourceforge.net/projects/renderfarm/
>      http://sourceforge.net/projects/littlefarm/
>      http://sourceforge.net/projects/barf/

Post a reply to this message

> also, the code in the server's main loop that waits for clients'
> requests and handles them, is *very* bad.  now that i have
> learned more, i'd do it with a simple state machine and fork()
> the handlers off (which won't work with windoze-perl :/ )...

Actually, NTFS supports any number of forks per file, although I still 
think that forking is a bad idea. :P

Post a reply to this message

> There are several existing projects that I know of out there.

When I started with SMPOV i found nothing that was as simple as I wanted it
for myself.
WEB-based management ? http-communication ? cross-platform ?
I did not need that, others would like to see that. I have only Windows
here.
Not for philosopical reasons (have no problem with Linux) but for practical
reasons (work).

What I needed was in short:
- drag a ".pov"-file onto a program
- the POV-Ray on all PC's start up rendering
- have the resulting picture in the same folder where the source file was
originally.
- 1,2,3
So this was my "basic thought". Want to share it at this place.

>it frame by frame.  Anything else gets really complicated, really buggy,
> really fast.

Buggy ? That maybe true if you use these "WEB-technologies" (maybe).
For me the tiling works fine for SMPOV so long you keep distance from scenes
that use
Radiosity or those sweet mechsims..
You cannot tile them or lets say "You can but the resulting
pictures/animations are just funny".

Basically the "tiling" is the only chance to save time using the "standard
unpatched POV 3.5"
rendering ONE single picture. So I can say it works really fine for me.
If you don't use tiling, you can "only" distribute frames.

Basically what SMPOV does looks like that:
- get a ".pov"-file
- put a copy of it into the "shared folder" (a folder where all the clients
have access to)
- put a "signal-file with a job-description there"
- start a thread that waits that this job is done

on the client side the render-agents do this:
- wait for "signal-file with a job-description there"
- read it and correct pathes (for the POV-Ray executable)
- start the render-job and place a resulting ".bmp" into the COM-folder
- delete the "signal-file with a job-description" and make ".rdy" file
instead to tell SMPOV "the tile is ready"

Now SMPOV (the thread ...) finds the ".rdy"-files,  counts them if ALL tiles
are ready. If so:
- starts PicPend.exe which "puts the tiles together to a single picture" and
also
- deletes all temp-files
- copies the resulting bmp-files to the original folder (where the
".pov"-fiile was "dropped from".

Other features are:
- distribution of frames (render without tiling)
- support for MEgaPOV 1.0

Currently there is no "controll-panel" to controll the clients from the
server. I have all my PC's in the same room so  I did not need that :-).

Thats in short how I wanted it and I could not find it so I did it myself.
Not WEB-based, cause I only wanted to render locally.

However I'd myself switch to any other sollution if it works better or is
easier. Just cannot
tell you at what point.

Oh ... and Ian Clark (See first post of this thread) got it running now !
And he sent me a new corrected text for SMPOV which is now included in the
package. Maybe you try it on anythig that does not end with "x" :-).

--Theo
----------------------------------------------------------------------------
---------------------------------
Distributed Network-Rendering or Local SMP-Rendering on all CPU's you have.
With SMPOV und
POV-Ray 3.5. & NEW: Mega-POV 1.0 * Download free at:
http://www.it-berater.org/smpov.htm

> There are several existing projects that I know of out there.  A summary
> of them is at the end of this reply.  I would suggest starting simple. Do
> it frame by frame.  Anything else gets really complicated, really buggy,
> really fast.  Based on the work I have done for the Internet Movie
> Project I have boiled the process down to this:
>
> The server just needs to do three things.
>
>     1) Issue frames
>     3) Track frames( Preferably in a database)
>     2) Receive frames
>
> The client just needs to do three things
>     1) Get a frame
>     2) Render a frame
>     3) Return a frame
>
> Our design goals are:
>
>     * Cross platform client/server
>     * http communication
>     * web based management
>
> Obviously the devil is in the details.  Both Gimli and IMP have made the
> source available if you want to see what has been done.  The server
> portion of IMPFarm is still two ASP pages.  They are being converted to
> perl.  If you want to join the IMPFarm project just sign up. You can see
> the work we have done so far if you look at
>
>    http://test.imp.org/impfarm/
>
>
> Tom
>
>
>
>
> POV-Ray RenderFarms
> =======================================
> IMPFarm
>     Language=Perl
>     Renderer=POV-Ray
>     http://sourceforge.net/projects/impfarm/
>
> Povray Render Server(Gimli already replied to this thread)
>      Language=Perl
>      Renderer=POV-Ray
>      http://www.psico.ch/index.php?page=prs
>
> The PVM patch for POV-Ray
>      Language = C++
>      Renderer = POV-Ray
>      http://pvmpov.sourceforge.net/
>
>
>
>
> Non POV-Ray projects
> =======================================
> BORG - BORG Open Rendering GUI
>      Language = Java
>      Renderer = Renderman compliant
>      http://www.project-borg.org/home/index.html
>
> RenderFarm.NET
>      Language = XML
>      Renderer = Maya
>      http://www.renderfarm3d.com/
>
> And these projects on source forge look dead:
>      http://sourceforge.net/projects/renderfarm/
>      http://sourceforge.net/projects/littlefarm/
>      http://sourceforge.net/projects/barf/
>
>
>
>

Post a reply to this message