POV-Ray : Newsgroups : povray.text.tutorials : jhu's very ghetto cluster for rendering still images Server Time
21 Nov 2024 12:35:59 EST (-0500)
  jhu's very ghetto cluster for rendering still images (Message 1 to 5 of 5)  
From: jhu
Subject: jhu's very ghetto cluster for rendering still images
Date: 25 Mar 2010 03:05:01
Message: <web.4bab09e0aad27b55ab8b233e0@news.povray.org>
If you're like me, you'll have several computers at your disposal that you've
collected over the years. Well, time to dust them off and make them do something
useful. Currently I have a 2.6 GHz Athlon64 and a 1.2 GHz Celeron-L at my
disposal.

Here are the general steps to set this whole thing up:

0) Put some sort of unix-like OS on the computers and install ssh (I'm assuming
you've already done this part)
1) Give all computers access to the same directory via nfs
2) Make a copy of the .pov file for each computer
3) Have each computer render one or more lines of the scene
4) stitch all the scene files together

Sounds simple, so let's get started.


PART 1 - Give all computers access to the same directory via nfs


If you don't already have a centralized file server with nfs, you can designate
one of computers as the nfs server. Grab the packages via your particular
package manager. Now to setup your nfs server; here's a good link:
http://nfs.sourceforge.net/nfs-howto/ar01s03.html

The gist of it is to edit "/etc/hosts.deny", "/etc/hosts.allow", and
"/etc/exports".

Let's edit "/etc/hosts.deny" first and put in the following:
portmap:ALL
lockd:ALL
mountd:ALL
rquotad:ALL
statd:ALL

That way no one can mount your nfs volume, even you! So let's edit
"/etc/hosts.allow" to change that. Here's mine:
portmap: 192.168.11.204, 192.168.1.204, 192.168.11.206
lockd:   192.168.11.204, 192.168.1.204, 192.168.11.206
rquotad: 192.168.11.204 , 192.168.1.204, 192.168.11.206
mountd:  192.168.11.204 , 192.168.1.204, 192.168.11.206
statd:   192.168.11.204 , 192.168.1.204, 192.168.11.206

You'll need to know your computers' IP addresses or host names. Then add them
in. Now to edit the "/etc/exports" file. Here's mine:
/opt 192.168.1.204(rw,insecure,sync,no_wdelay)
/opt 192.168.11.204(rw,insecure,sync,no_wdelay)
/opt 192.168.11.206(rw,insecure,sync,no_wdelay)

In my case, I'm sharing my nfs server's "/opt" directory with the other
computers. Now you just need to mount the remote directory to your computer
somewhere. I made a new directory "nfs" in my other computers' home directory so
when I remotely log in to them I just type "mount 192.168.11.101:/opt nfs" with
192.168.11.101 being my nfs server


PART 2

So now you have the remote directory mounted on each computer. Now to make
multiple copies of your .pov file. Let's call them scene1.pov, scene2.pov, and
scene3.pov. If you need radiosity, you'll need to render the original scene and
edit your scene file to save the radiosity data. Once that is done, edit all the
copies of the scene files to load the radiosity data. Now the fun part. Let's
write a script in the shared directory that will be run by each computer.

Here's script1.sh :

#!/bin/bash
while [ ! -e "scene_1080.ppm" ]
do
 for ((i=1; i<=1080; i++))
 do
  if [ ! -e "scene_$i.ppm" ]
   then
    touch scene_$i.ppm
    povray +A0.3 -d +FP +iscene1 +Oscene_$i.ppm +w1920 +h1080 +SR$i +ER$i
  fi
 done
done


Here's script2.sh :

#!/bin/bash
while [ ! -e "scene_1080.ppm" ]
do
 for ((i=1; i<=1080; i++))
 do
  if [ ! -e "scene_$i.ppm" ]
   then
    touch scene_$i.ppm
    povray +A0.3 -d +FP +iscene2 +Oscene_$i.ppm +w1920 +h1080 +SR$i +ER$i
  fi
 done
done

The preceding scripts render the scene at 1920x1080 one line at a time.
Eventually we'll end up with 1080 PPM files of 1920 pixels in length. Now,
depending on how long it takes to parse the scene file, it may be better to make
POV-Ray render several rows at a time instead of one at a time. Edit as
appropriate.


PART 3

So now we have 1080 PPM files. PPM files are great for manipulation since the
way POV-Ray generates them, the first three lines are text followed by the image
data. In general there are 6 types of PPM files, designated P1, P2, P3, P4, P5,
or P6. POV-Ray outputs PPM files as the P6 type: the data portion is in red,
green, blue triplets in binary format. The other significant one is the P3 type
in which the RGB triplets are in ASCII.

If you open up one of these files in a text editor you'll see the following:

P6
1920 1080
255
*** image data ***

So what we need to do is make a new PPM file with an appropriate header,
strip the first three lines from each of our 1080 PPM files, and then append the
image data to the new file. Here's the script to do that:

#!/bin/sh
echo "P6 1920 1080 255" >> scene.ppm

for ((i=1; i<=1080; i++))
do
 cat scene_$i.ppm | sed '1,3d' >> scene.ppm
done


Now you can convert scene.ppm to your favorite image format.


Post a reply to this message

From: Warp
Subject: Re: jhu's very ghetto cluster for rendering still images
Date: 25 Mar 2010 03:31:36
Message: <4bab1158@news.povray.org>
jhu <nomail@nomail> wrote:
> #!/bin/sh
> echo "P6 1920 1080 255" >> scene.ppm

> for ((i=1; i<=1080; i++))

  AFAIK that form of the 'for' directive is a bash extension not available
in the original sh. Hence that script is basically broken because the first
line claims it's runnable by using sh.

  (In linux systems /bin/sh is just an alias for /bin/bash, which is something
I cannot comprehend. bash is certain *not* the same thing as sh, and making
them equal only produces broken sh scripts which assume bash extensions but
still claim to be runnable using sh. The script above works, but only if if
/bin/sh is an alias for bash or another shell that supports the extension.
It won't work in systems where /bin/sh is the true original sh.)

>  cat scene_$i.ppm | sed '1,3d' >> scene.ppm

  sed '1,3d' scene_$i.ppm >> scene.ppm

-- 
                                                          - Warp


Post a reply to this message

From: jhu
Subject: Re: jhu's very ghetto cluster for rendering still images
Date: 25 Mar 2010 04:15:01
Message: <web.4bab1b50a3514f79ab8b233e0@news.povray.org>
Warp <war### [at] tagpovrayorg> wrote:
> jhu <nomail@nomail> wrote:
> > #!/bin/sh
> > echo "P6 1920 1080 255" >> scene.ppm
>
> > for ((i=1; i<=1080; i++))
>
>   AFAIK that form of the 'for' directive is a bash extension not available
> in the original sh. Hence that script is basically broken because the first
> line claims it's runnable by using sh.
>
>   (In linux systems /bin/sh is just an alias for /bin/bash, which is something
> I cannot comprehend. bash is certain *not* the same thing as sh, and making
> them equal only produces broken sh scripts which assume bash extensions but
> still claim to be runnable using sh. The script above works, but only if if
> /bin/sh is an alias for bash or another shell that supports the extension.
> It won't work in systems where /bin/sh is the true original sh.)
>
> >  cat scene_$i.ppm | sed '1,3d' >> scene.ppm
>
>   sed '1,3d' scene_$i.ppm >> scene.ppm
>
> --
>                                                           - Warp

Never noticed that since I always installed bash everywhere I went. Hehe...


Post a reply to this message

From: Kenneth
Subject: Re: jhu's very ghetto cluster for rendering still images
Date: 25 Mar 2010 18:20:01
Message: <web.4babe028a3514f7965f302820@news.povray.org>
"jhu" <nomail@nomail> wrote:
> If you're like me, you'll have several computers at your disposal that you've
> collected over the years. Well, time to dust them off and make them do something
> useful...
>
> Here are the general steps to set this whole thing up:
> .....
> Sounds simple, so let's get started.
>

Good God Almighty.

I think you left one step out (at least for me):

00) Return to college, get Computer Science degree.

Ken  :-[


Post a reply to this message

From: jhu
Subject: Re: jhu's very ghetto cluster for rendering still images
Date: 25 Mar 2010 18:45:01
Message: <web.4babe670a3514f79ab8b233e0@news.povray.org>
"Kenneth" <kdw### [at] earthlinknet> wrote:
> "jhu" <nomail@nomail> wrote:
> > If you're like me, you'll have several computers at your disposal that you've
> > collected over the years. Well, time to dust them off and make them do something
> > useful...
> >
> > Here are the general steps to set this whole thing up:
> > .....
> > Sounds simple, so let's get started.
> >
>
> Good God Almighty.
>
> I think you left one step out (at least for me):
>
> 00) Return to college, get Computer Science degree.
>
> Ken  :-[

This coming from the guy who made the B29 bomber over here:
http://news.povray.org/povray.binaries.images/thread/%3Cweb.4baae0a4649f667b65f302820%40news.povray.org%3E/

I think my ghetto cluster setup is pretty simple relatively speaking!


Post a reply to this message

Copyright 2003-2023 Persistence of Vision Raytracer Pty. Ltd.