I have been a POV-Ray fan for a long time. I know there have been several
approaches to distributed rendering over the years, but after retiring I was
looking for a project and decided to create my own distributed client-server
rendering system, using POV-Ray 3.7.0.10 as the base.

So far the results have been encouraging. With 8 clients, measured speedup for
still-image rendering ranges from 88.4% to 98.2% scaling efficiency, depending
on image size and complexity (larger, more complex images scale best). Animation

depending on scene characteristics. These gains come not only from distributing
work across clients, but also from parser and caching optimizations that
eliminate a significant amount of repeated work.

Here is a summary of the project.

For hardware, I wanted to see what could be accomplished with a cluster of
Raspberry Pi 5 Compute Modules. To that end I built two self-contained four-node
Pi clusters. I'm happy to share hardware details if anyone is interested, but
the hardware is not really the focus of this post.

The more interesting work was creating the client-server software environment.
Starting with POV-Ray 3.7.0.10, I modified the renderer to operate as a
non-terminating client and optimized it for distributed operation. The server
assigns work and collects results. Work distribution itself is straightforward:
still images are divided into row ranges, while animations are distributed one
frame per client.

The key client-side modifications are aimed at avoiding unnecessary repeated
work. Each client maintains a single persistent POV-Ray session across multiple
tasks and renders. Parsed scene information is cached so that repeated renders
of the same scene do not require reparsing. In addition, I modified the parser
to support a static keyword, allowing large clock-independent structures to be
cached and reused between animation frames. Similar concepts have been explored
before, but the persistent-session architecture makes this particularly
effective.

I also added a feature that I personally felt was missing from POV-Ray: the
ability to call precompiled native functions directly from the Scene Description
Language. To support this, I developed a plugin system and extended the parser
with five new function types: userfun(), uservectfun(), usercolorfun(),
useruv(), and userfracfun(), returning scalar, vector, color, UV, and 4D values
respectively.
For example:
uservectfun("curl_field", x, y, z)
where curl_field is a function exported by a dynamically loaded plugin.

The server side includes a number of supporting capabilities, including
heartbeat monitoring, failed-task reassignment, thermal management, archive
reuse, and delta synchronization of include files.

I'd be interested in hearing whether others have experimented with similar
persistent-session approaches, parser extensions, or SDL plugin mechanisms.

Post a reply to this message

"boltcapt" <bol### [at] aolcom> wrote:

the blank 9th line should read:

performance is even more interesting: total speedups range from 17x to 63x,

I have no idea why it was deleted

> I have been a POV-Ray fan for a long time. I know there have been several
> approaches to distributed rendering over the years, but after retiring I was
> looking for a project and decided to create my own distributed client-server
> rendering system, using POV-Ray 3.7.0.10 as the base.
>
> So far the results have been encouraging. With 8 clients, measured speedup for
> still-image rendering ranges from 88.4% to 98.2% scaling efficiency, depending
> on image size and complexity (larger, more complex images scale best). Animation
>
> depending on scene characteristics. These gains come not only from distributing
> work across clients, but also from parser and caching optimizations that
> eliminate a significant amount of repeated work.
>
> Here is a summary of the project.
>
> For hardware, I wanted to see what could be accomplished with a cluster of
> Raspberry Pi 5 Compute Modules. To that end I built two self-contained four-node
> Pi clusters. I'm happy to share hardware details if anyone is interested, but
> the hardware is not really the focus of this post.
>
> The more interesting work was creating the client-server software environment.
> Starting with POV-Ray 3.7.0.10, I modified the renderer to operate as a
> non-terminating client and optimized it for distributed operation. The server
> assigns work and collects results. Work distribution itself is straightforward:
> still images are divided into row ranges, while animations are distributed one
> frame per client.
>
> The key client-side modifications are aimed at avoiding unnecessary repeated
> work. Each client maintains a single persistent POV-Ray session across multiple
> tasks and renders. Parsed scene information is cached so that repeated renders
> of the same scene do not require reparsing. In addition, I modified the parser
> to support a static keyword, allowing large clock-independent structures to be
> cached and reused between animation frames. Similar concepts have been explored
> before, but the persistent-session architecture makes this particularly
> effective.
>
> I also added a feature that I personally felt was missing from POV-Ray: the
> ability to call precompiled native functions directly from the Scene Description
> Language. To support this, I developed a plugin system and extended the parser
> with five new function types: userfun(), uservectfun(), usercolorfun(),
> useruv(), and userfracfun(), returning scalar, vector, color, UV, and 4D values
> respectively.
> For example:
> uservectfun("curl_field", x, y, z)
> where curl_field is a function exported by a dynamically loaded plugin.
>
> The server side includes a number of supporting capabilities, including
> heartbeat monitoring, failed-task reassignment, thermal management, archive
> reuse, and delta synchronization of include files.
>
> I'd be interested in hearing whether others have experimented with similar
> persistent-session approaches, parser extensions, or SDL plugin mechanisms.

Post a reply to this message

That question has been answered as recently as this thread:

https://news.povray.org/povray.off-topic/message/%3Cweb.6a23f45069259b1fd1d09a9825979125%40news.povray.org%3E/#%3Cweb.6
a23f45069259b1fd1d09a9825979125%40news.povray.org%3E

Just don't use any UTF-8 characters and all should be fine.

- BE

Post a reply to this message

"boltcapt" <bol### [at] aolcom> wrote:

> I also added a feature that I personally felt was missing from POV-Ray: the
> ability to call precompiled native functions directly from the Scene Description
> Language. To support this, I developed a plugin system and extended the parser
> with five new function types: userfun(), uservectfun(), usercolorfun(),
> useruv(), and userfracfun(), returning scalar, vector, color, UV, and 4D values
> respectively.
> For example:
> uservectfun("curl_field", x, y, z)
> where curl_field is a function exported by a dynamically loaded plugin.

The other stuff sounds great, but plugins sound fantastic!

I am VERY eager to hear more.

Also, you seem to have a highly desirable skill set.
I have done an extensive amount of work tracking down some issues with source
code, but now find myself with limited time to get that implemented in a patch.

https://news.povray.org/povray.documentation.inbuilt/thread/%3Cweb.68bdae61251da1bc1f9dae3025979125%40news.povray.org%3
E/

Do you think that you could possibly help me get that work pushed across the
finish line so that all the time and effort was not wasted?

- BE

Post a reply to this message

"Bald Eagle" <cre### [at] netscapenet> wrote:
> "boltcapt" <bol### [at] aolcom> wrote:
>
> > I also added a feature that I personally felt was missing from POV-Ray: the
> > ability to call precompiled native functions directly from the Scene Description
> > Language. To support this, I developed a plugin system and extended the parser
> > with five new function types: userfun(), uservectfun(), usercolorfun(),
> > useruv(), and userfracfun(), returning scalar, vector, color, UV, and 4D values
> > respectively.
> > For example:
> > uservectfun("curl_field", x, y, z)
> > where curl_field is a function exported by a dynamically loaded plugin.
>
> The other stuff sounds great, but plugins sound fantastic!
>
> I am VERY eager to hear more.
>
> Also, you seem to have a highly desirable skill set.
> I have done an extensive amount of work tracking down some issues with source
> code, but now find myself with limited time to get that implemented in a patch.
>
>
https://news.povray.org/povray.documentation.inbuilt/thread/%3Cweb.68bdae61251da1bc1f9dae3025979125%40news.povray.org
%3
> E/
>
> Do you think that you could possibly help me get that work pushed across the
> finish line so that all the time and effort was not wasted?
>
> - BE

Happy to provide you with a good sample of a plugin package. It's long, but I
can email it.

Post a reply to this message

hi,

"boltcapt" <bol### [at] aolcom> wrote:
> "Bald Eagle" <cre### [at] netscapenet> wrote:
> > "boltcapt" <bol### [at] aolcom> wrote:
> > ...
> > > with five new function types: userfun(), uservectfun(), usercolorfun(),
> > > useruv(), and userfracfun(), returning scalar, vector, color, UV, and 4D values
> > > respectively.
> > > For example:
> > > uservectfun("curl_field", x, y, z)
> > > where curl_field is a function exported by a dynamically loaded plugin.
> >
> > The other stuff sounds great, but plugins sound fantastic!
> >
> > I am VERY eager to hear more.
> >
> > Also, you seem to have a highly desirable skill set.
> > I have done an extensive amount of work tracking down some issues with source
> > code, but now find myself with limited time to get that implemented in a patch.
> >
> >
https://news.povray.org/povray.documentation.inbuilt/thread/%3Cweb.68bdae61251da1bc1f9dae3025979125%40news.povray.o
rg
> %3
> > E/
> >
> > Do you think that you could possibly help me get that work pushed across the
> > finish line so that all the time and effort was not wasted?
> >
> > - BE
>
> Happy to provide you with a good sample of a plugin package. It's long, but I
> can email it.

could you also be interested in putting together a wiki page[*] to document your
work ?  tia.

[*] MediaWiki, would be happy to assist with markup etc if you provide
"content".


regards, jr.

Post a reply to this message