Website:
http://www.gammaburst.net/xrs

Improvements:

With valuable feedback from Jaime Vives Piqueres, I fixed a bug in version
2.0.5 where radiosity data was lost in between rendering tiles. Now there
are far fewer tiling artifacts when doing radiosity scenes, and they can be
eliminated altogether using Jaime's two-pass technique (involves
precalculating rad data using a normal version of megapov first)

Issues:

There are still issues with library path order being worked out. Currently
the client does not put the scene dir first, which can lead to problems in
some situations. A solution is being worked out. While the user can edit
the pyxrsclient.py file (it's a python script with GUI), the next version
will make it easier to configure the paths.

Comments and feedback appreciated as always

George

Post a reply to this message

George Pantazopoulos wrote:
> Website:
> http://www.gammaburst.net/xrs

I am strongly missing some explanation on how your tools and especially 
your modifications of POV-Ray work.  What is the server?  What is the 
client?  What data is exchanged between both and how (the screenshot 
indicates there is HTTP involved)?  What is different about the modified 
version of POV-Ray/MegaPOV and how does it interact with the scripts in 
the package?

This looks like a good approach on network rendering with POV-Ray 3.6 
(although the dependency on nonstandard libs is limiting) but some solid 
documentation would be very important.

BTW you should include the POV-Ray license files with the modified 
POV-Ray source files.

> With valuable feedback from Jaime Vives Piqueres, I fixed a bug in version
> 2.0.5 where radiosity data was lost in between rendering tiles. Now there
> are far fewer tiling artifacts when doing radiosity scenes, and they can be
> eliminated altogether using Jaime's two-pass technique (involves
> precalculating rad data using a normal version of megapov first)

Now this is unlikely - considering the modifications you made even 
impossible - that you can guarantee tiling artefact free results with 
non-sequential rendering.  Running the pretrace for the full image in 
advance will reduce the problems but it will not remove them (try 
comparing two images of two different renders with different tilings and 
you will see).

The newest radiosity patches in MegaPOV can help you minimize the 
differences but without further changes (which will involve performance 
loss) you won't get completely reproducible results in parallel renders 
using radiosity.

Christoph

-- 
POV-Ray tutorials, include files, Landscape of the week:
http://www.imagico.de/ (Last updated 20 Aug. 2006)
MegaPOV with mechanics simulation: http://megapov.inetart.net/

Post a reply to this message

> George Pantazopoulos wrote:
> > Website:
> > http://www.gammaburst.net/xrs
>
> I am strongly missing some explanation on how your tools and especially
> your modifications of POV-Ray work.  What is the server?  What is the
> client?  What data is exchanged between both and how (the screenshot
> indicates there is HTTP involved)?  What is different about the modified
> version of POV-Ray/MegaPOV and how does it interact with the scripts in
> the package?
>
> This looks like a good approach on network rendering with POV-Ray 3.6
> (although the dependency on nonstandard libs is limiting) but some solid
> documentation would be very important.

I think your questions are important, and I agree there needs to be more
solid documention. I'll put some effort towards that.

The "Server" is a modified version of MegaPOV, to which I added a new thread
to handle RPC requests.

The RPC itself is done with the simple XML-RPC protocol over HTTP. The open
source library libiqxmlrpc implements the XML-RPC interface. There is no
security layer, but the use of ssh with port forwarding will suffice and is
strongly recommended for internet renders.

There was an active synergy between the development of Megapov-XRS and
libiqxmlrpc. My feedback to the developer has helped libiqxmlrpc become a
more solid and polished product.

Main modifications of the XRS_PATCH to MegaPOV:
-----------------------------------------------
- New thread added to handle RPC requests over a socket connection.

- "Command line" arguments are taken from the RPC thread instead of the
console.

- Certain scene initializations such as parsing and photon shooting are only
done once per scene session, and not for every tile.


- A NewScene() request aborts the current scene and starts a new one.

- Code was added to gracefully handle errors and abort requests. Instead of
  quitting the program they now simply terminate the current session and
megapov_xrs_server goes back to waiting for a NewScene() request over RPC.

- Code was added to allow photon shooting to be aborted quickly.

There are a few more updates, but that's in a nutshell what I modified.

pyxrsclient
-----------

The client is a python script which conrols the servers and coordinates the
parallel render. The servers are completely passive right now, so its up to
the client to query the servers as to their status. pyxrsclient

The client was a lot of fun to develop mostly because I really love
programming in Python. I was thrilled to find that Python comes with
xmlrpclib, which I used right of the box to communicate with
megapov_xrs_server.

>
> BTW you should include the POV-Ray license files with the modified
> POV-Ray source files.
>

Will do, thanks.

> > With valuable feedback from Jaime Vives Piqueres, I fixed a bug in version
> > 2.0.5 where radiosity data was lost in between rendering tiles. Now there
> > are far fewer tiling artifacts when doing radiosity scenes, and they can be
> > eliminated altogether using Jaime's two-pass technique (involves
> > precalculating rad data using a normal version of megapov first)
>
> Now this is unlikely - considering the modifications you made even
> impossible - that you can guarantee tiling artefact free results with
> non-sequential rendering.  Running the pretrace for the full image in
> advance will reduce the problems but it will not remove them (try
> comparing two images of two different renders with different tilings and
> you will see).
>

That's right, I can't guarantee renders free of tiling artifacts with
MegaPOV XRS alone. However, what Jaime does is that he uses a normal
version of megapov to create the radiosity data ahead of time. Then he
modifies the scene with "always_sample off" and "load_file". Then he used
Megapov XRS to render this scene, loading only the precalculated rad data
and minimizing the calculation of new rad data while in parallel.

> The newest radiosity patches in MegaPOV can help you minimize the
> differences but without further changes (which will involve performance
> loss) you won't get completely reproducible results in parallel renders
> using radiosity.
>

Which radiosity patches are those? Sounds interesting...

Thanks for your feedback, Christoph

Regards,
George

Post a reply to this message

Christoph Hormann wrote:
> Now this is unlikely - considering the modifications you made even 
> impossible - that you can guarantee tiling artefact free results with 
> non-sequential rendering.  Running the pretrace for the full image in 
> advance will reduce the problems but it will not remove them (try 
> comparing two images of two different renders with different tilings and 
> you will see).

   I tested it now with several scenes, and I don't see any tiling
artifact when using precalculated radiosity. See the examples on p.b.i.
The rad data was saved with MegaPOV on the fastest computer, and copied
to the other computers (which already had all the scene files and the
same povray config) for the parallel render.

--
Jaime

Post a reply to this message

> BTW you should include the POV-Ray license files with the modified
> POV-Ray source files.
>

I already include the original "AUTHORS", "COPYING", "ChangeLog", "INSTALL",
"NEWS", and "README" files in the megapov-xrs_core distribution. They're one
directory level down, inside megapov-xrs_core/xrs_core, however. Is that not
adequate?

Thanks,
George

Post a reply to this message

George Pantazopoulos wrote:
> I already include the original "AUTHORS", "COPYING", "ChangeLog", "INSTALL",
> "NEWS", and "README" files in the megapov-xrs_core distribution. They're one
> directory level down, inside megapov-xrs_core/xrs_core, however. Is that not
> adequate?

   I think you should include the "doc" folder, which contains the 
documents referenced on "COPYING".

--
Jaime

Post a reply to this message

Jaime Vives Piqueres <jai### [at] ignoranciaorg> wrote:
> George Pantazopoulos wrote:
> > I already include the original "AUTHORS", "COPYING", "ChangeLog", "INSTALL",
> > "NEWS", and "README" files in the megapov-xrs_core distribution. They're one
> > directory level down, inside megapov-xrs_core/xrs_core, however. Is that not
> > adequate?
>
>    I think you should include the "doc" folder, which contains the
> documents referenced on "COPYING".
>
> --
> Jaime

Doh! That little guy was almost invisible :) Done. Thanks!

George

Post a reply to this message

George Pantazopoulos wrote:
> 
> I think your questions are important, and I agree there needs to be more
> solid documention. I'll put some effort towards that.
> 
> The "Server" is a modified version of MegaPOV, to which I added a new thread
> to handle RPC requests.

So the server needs to run on all nodes supposed to participate in the 
distributed render.  I suppose when no render is running it is an idle 
program.

> 
> - Certain scene initializations such as parsing and photon shooting are only
> done once per scene session, and not for every tile.

But how do you transfer the data to the other nodes in a portable way 
then (and one that does not eat tons of network bandwidth)?

> pyxrsclient
> -----------
> 
> The client is a python script which conrols the servers and coordinates the
> parallel render. The servers are completely passive right now, so its up to
> the client to query the servers as to their status. pyxrsclient
> 
> The client was a lot of fun to develop mostly because I really love
> programming in Python. I was thrilled to find that Python comes with
> xmlrpclib, which I used right of the box to communicate with
> megapov_xrs_server.

I would suggest you sepearate the client from the GUI - a lot of people 
will probably prefer a command line interface (preferably syntax 
compatible with POV-Ray) or use it in a CGI script.

> 
> That's right, I can't guarantee renders free of tiling artifacts with
> MegaPOV XRS alone. However, what Jaime does is that he uses a normal
> version of megapov to create the radiosity data ahead of time. Then he
> modifies the scene with "always_sample off" and "load_file". Then he used
> Megapov XRS to render this scene, loading only the precalculated rad data
> and minimizing the calculation of new rad data while in parallel.

Well - the key word is 'minimize' - by design POV-Ray radiosity does 
always take new samples when necessary even in the final pass. 
Depending on scene and settings this can be a lot.

>> The newest radiosity patches in MegaPOV can help you minimize the
>> differences but without further changes (which will involve performance
>> loss) you won't get completely reproducible results in parallel renders
>> using radiosity.
>>
> 
> Which radiosity patches are those? Sounds interesting...

RADIOSITY_ADAPTIVE_RADIUS_PATCH and NEW_RADIOSITY_PRETRACE_PATCH.

Christoph

-- 
POV-Ray tutorials, include files, Landscape of the week:
http://www.imagico.de/ (Last updated 20 Aug. 2006)
MegaPOV with mechanics simulation: http://megapov.inetart.net/

Post a reply to this message

Jaime Vives Piqueres wrote:
> Christoph Hormann wrote:
>> Now this is unlikely - considering the modifications you made even 
>> impossible - that you can guarantee tiling artefact free results with 
>> non-sequential rendering.  Running the pretrace for the full image in 
>> advance will reduce the problems but it will not remove them (try 
>> comparing two images of two different renders with different tilings 
>> and you will see).
> 
>   I tested it now with several scenes, and I don't see any tiling
> artifact when using precalculated radiosity. See the examples on p.b.i.
> The rad data was saved with MegaPOV on the fastest computer, and copied
> to the other computers (which already had all the scene files and the
> same povray config) for the parallel render.

I am pretty sure even if you can't see any differences you will be able 
see them when actually calculating the difference between the images 
(can't well do that with your example since it is jpg-compressed).  The 
difference might be small in a lot of cases but with critical settings 
it can be large enough to be visible (in the extreme case you don't do 
any pretrace like Gilles for example did in a lot of cases).

Christoph

-- 
POV-Ray tutorials, include files, Landscape of the week:
http://www.imagico.de/ (Last updated 20 Aug. 2006)
MegaPOV with mechanics simulation: http://megapov.inetart.net/

Post a reply to this message

Christoph Hormann wrote:
> I am pretty sure even if you can't see any differences you will be able 
> see them when actually calculating the difference between the images 
> (can't well do that with your example since it is jpg-compressed).  The 
> difference might be small in a lot of cases but with critical settings 
> it can be large enough to be visible (in the extreme case you don't do 
> any pretrace like Gilles for example did in a lot of cases).

   You are right, the stats report about 1-2% of new samples. But almost 
in all my recent scenes this leads to invisible artifacts. On the 
difference image you have to raise brightness and contrast to the 
maximum to see them. But curiously, the artifacts do not seem to 
visually have any relation with the tiles... they seem a global thing.

   In any case, I suppose this happens also when I distribute manually a 
rendering over my machines. And as I always use the two-pass radiosity 
method, the only difference for me when using XRS is that I need much 
less effort and time to obtain the same result.

--
Jaime

Post a reply to this message