To give an overview of current development progress:

* Some differences to 3.6 have been identified and reverted.

* Potential multithreading issues in the radiosity code have been examined; the
beta.29 code was found to be technically thread-safe regarding access to the
radiosity data cache.

The locking concept has been refined to improve performance.

The known issues regarding exact reproducability of output have not been
addressed yet (except that a basic idea exists). Currently, exact
reproducability of output of radiosity scenes can only be guaranteed for
single-threaded operation, and provided that the same version is used.

* Output and performance of the current development version have been compared
to 3.6 with satisfactory results.

The following scenes were used as benchmark samples:

- All scenes in ../scenes/radiosity/
- All scenes in ../scenes/advanced/ using radiosity
- Two self-made scenes

Resolution was set to the highest recommended values from the respective scene
files. Animations were not tested.

Generally, all scenes now render virtually identical to the 3.6 output (showing
only random jitter effects), except for the following:

- ../scenes/radiosity/cornell.pov: The top-rear corners of the room render
slightly darker, while the top-front-right edge of the cube renders slightly
brighter; these may still be just statistical differences though

- ../scenes/advanced/balcony/: Some artifact-ish speckles in the roof seen in
the 3.6 shot are eliminated; the floor renders significantly brighter
especially in the shadows; the table cloth renders significantly darker in the
shadows (almost black); however, the complexity of the scene currently prevents
a thorough analysis of the reasons.

- ../scenes/advanced/blocks/: The scenes render brighter due to different gamma
handling.

- ../scenes/advanced/object_pattern.pov: This scene shows a significant
brightness gain in darker areas, and some "artifacts" in the same areas; closer
inspection shows that the "artifacts" are actually reflections of shadows, and
that the brightening is due to the non-shadowed areas being reflected. So we
actually have an improvement in quality, but it also hints at some yet
undiscovered difference to 3.6 code.

- The self-made scenes showed significant differences not yet analyzed. One of
the scenes indicates a higher robustness against undesired "light leaks", but
the reason for this is not explained yet, and in any way indicates unidentified
differences to 3.6 code.


Runtime behaviour was compared on an AMD Phenom X4 9650 2.3GHz QuadCore with 6
GB of RAM running a 64-bit Linux. MegaPOV 1.2.1 was used instead of POV 3.6 due
to availability of a better-optimized version.

Pure computing power requirements ("CPU time") varied within ca. +/-20% of that
of MegaPOV. Comparison of CPU time with "Wall clock" time showed typical "dead"
time of 1-2% when running 4 threads for larger scenes, going as high as 5% for
one of the self-made scenes. Smaller scenes, running just a few seconds,
typically showed a significantly higher "dead" time, maybe due a
disproportionate effect of I/O delays during scene parsing.

Post a reply to this message

clipka <nomail@nomail> wrote:
> reproducability

  I really hate to sound like a grammar nazi, but you consistently keep
writing it like that, and it hurts my eyes. It's "reproducibility".

  Btw, personally I would be more concerned about getting all threads
produce the same lighting for all the rendered parts, than trying to
make POV-Ray reproduce bit-by-bit the exact same image each time it
renders the same scene. The biggest problem with "radiosity" in POV-Ray
has always been that there's no way to distribute it because if you make
different instances render different parts of the program, the "radiosity"
lighting will be calculated differently in them, and a visible difference
will appear between the image sections rendered by different instances.

-- 
                                                          - Warp

Post a reply to this message

Warp <war### [at] tagpovrayorg> wrote:
> The biggest problem with "radiosity" in POV-Ray
> has always been that there's no way to distribute it because if you make
> different instances render different parts of the program, the "radiosity"
> lighting will be calculated differently in them, and a visible difference
> will appear between the image sections rendered by different instances.

It seems this problem is related to the one that the calculation only takes into
account things in sight, right?  Like the problem I mentioned before of having
to calculate things again if the scene if shot from another angle and despite a
loaded rad file.

Ward's paper mentions his algorithm is meant for view independence, but only to
an extent.  Here's what he says on the subject, from page 2:

http://radsite.lbl.gov/radiance/papers/sg88/paper.html (the gifs have diagrams)

"For the sake of efficiency, indirect illuminance should not be recalculated at
each pixel, but should instead be averaged over surfaces from a small set of
computed values.  Computing each value might require many samples, but the
number of values would not depend on the number of pixels, so high resolution
images could be produced efficiently.  Also, since illuminance does not depend
on view, the value could be reused for many images."

So far, so good:  pixel and view independence mean reusing the same saved
calculations for any resolution and also from any camera angles and different
positions.  But on the next paragraph, the latter is compromised:

"How can we benefit from a view-independent calculation in the inherently
view-dependent world of ray tracing?  We do not wish to limit or burden the
geometric model with illuminance information, as required by the surface
discretization of the radiosity method.  By the same token, we do not wish to
take view-independence too far, calculating illuminance on surfaces that play
no part in the desired view.  Instead, we would like to take our large sample
of rays only when and where it is necessary for the accurate computation of an
image, storing the result in a separate data structure that puts no constraints
on the surface geometry."

He's a performance couscious guy and did tackle Kajiya's path tracing method in
the first page.

It seems this feature, while a nice optimization trick, is what prevents one
from reusing a full saved rad file to render a file from a different angle
without taking any more samples and probably also the root cause of not being
able to simply break an image into sections, render them separately and simply
merge the final renders into a single image later.  Each instance do not know
they should be taking samples from areas not seen in their particular view.

Perhaps this optimization was powerful then, but not quite as fitting in an era
of multicore processors and parallel processing.  Would it be possible to
simply remove this view-dependent constraints and allow for full view
independence in the current implementation?  How bad would performance drop?

Post a reply to this message

Warp <war### [at] tagpovrayorg> wrote:
>   I really hate to sound like a grammar nazi, but you consistently keep
> writing it like that, and it hurts my eyes. It's "reproducibility".

Sorry - I keep wondering what the correct spelling actually is every time I type
that word (btw, to do some nitpicking myself: It's not a grammar issue, but a
matter of orthography... I hope I got this right ;)).

BTW, I'm German (hence those orthography issues)...

>   Btw, personally I would be more concerned about getting all threads
> produce the same lighting for all the rendered parts, than trying to
> make POV-Ray reproduce bit-by-bit the exact same image each time it
> renders the same scene. The biggest problem with "radiosity" in POV-Ray
> has always been that there's no way to distribute it because if you make
> different instances render different parts of the program, the "radiosity"
> lighting will be calculated differently in them, and a visible difference
> will appear between the image sections rendered by different instances.

The threads do co-operate well in this respect. There have been probems with the
octree structure showing in some shots, and artifacts related to the "job
chunks", but these have been solved. (In fact the octree artifacts were totally
unrelated to the multithreading, and the other artifacts were just made more
prominent by the block-by-block rendering.)

I'm not sure how the POV team thinks about this topic, but as far as I'm
concerned I'd have no problem with releasing the current status as a beta.

Post a reply to this message

"nemesis" <nam### [at] gmailcom> wrote:
> It seems this feature, while a nice optimization trick, is what prevents one
> from reusing a full saved rad file to render a file from a different angle
> without taking any more samples and probably also the root cause of not being
> able to simply break an image into sections, render them separately and simply
> merge the final renders into a single image later.  Each instance do not know
> they should be taking samples from areas not seen in their particular view.
>
> Perhaps this optimization was powerful then, but not quite as fitting in an era
> of multicore processors and parallel processing.  Would it be possible to
> simply remove this view-dependent constraints and allow for full view
> independence in the current implementation?  How bad would performance drop?

If you have a scene containing a complete 10-storey house, and you only want to
render a single room - it'll kill.

It's an extreme example, but it shows that (a) your question cannot be answered
without knowing the scene, and how much of it is actually visible, and (b)
processing power has not increased so much to just go for plain brute force.

In fact, instead of spending the processing power to speed up *potential* later
renders of other parts of the scene, I'd rather see the added power invested in
(a) more detail, (b) higher quality and (c) faster results.

Actually, in an ideal world, the view-based approach of Ward's algorithm should
still prove superior to a pre-render of the whole scene, regardless of scene
size. It would be as follows:

- First image is pretraced, producing all necessary samples for the first shot
- First image is rendered, no additional samples are needed
- Second image is pretraced, re-using what the first pretrace produced, and
adding what is missing
- Second image is rendered, no additional samples are needed
- ...

The only problems here are...
(a) that the pretraces often do *not* produce all the necessary samples for some
reason;
(b) that the settings do not produce good enough quality; so the quality
increases from shot to shot (because for some parts they have more samples
available than they would actively collect themselves), but people think the
algorithm just produces randomly different results (partly because it is not
easy to asses which picture is actually more accurate);
(c) that the pretrace step may be done wrong, if a separate render is used; for
example, if you use micronormals, it is actually a bad idea to disable them for
the pretrace, because it causes the pretrace to collect less (and insufficient)
samples; likewise, reducing the recursion depth during pretrace may cause
additional samples to be required during the main render. Probably the only
quality setting that can safely be disabled is area light resolution (though
this could cause artifacts if additional samples are required during main
render for other reasons).

Post a reply to this message

"clipka" <nomail@nomail> wrote:
> "nemesis" <nam### [at] gmailcom> wrote:
> If you have a scene containing a complete 10-storey house, and you only want to
> render a single room - it'll kill.
>
> It's an extreme example, but it shows that (a) your question cannot be answered
> without knowing the scene, and how much of it is actually visible, and (b)
> processing power has not increased so much to just go for plain brute force.

I'd never put a full building to render when only targetting a room.  It's up to
the user to render their scenes in little passes and then combine them -- or
not, in this case.  Perhaps some sort of "radiosity bounding box" could help
here?

> In fact, instead of spending the processing power to speed up *potential* later
> renders of other parts of the scene, I'd rather see the added power invested in
> (a) more detail, (b) higher quality and (c) faster results.

It seems to me faster results are dependent on parallelization of the
"radiosity" calculation step and such step is being hampered of achieving such
parallelism exactly by not being able to know in advance how much each thread
should allow to "be visible".  At least in the case of different instances of
povray rendering chunks of a larger image... as was the old approach to
parallelism.  Not sure threads would improve the situation here, possibly yes,
by having shared datastructures.  OTOH, I'm aware of the locking issues...

> - First image is pretraced, producing all necessary samples for the first shot
> - First image is rendered, no additional samples are needed
> - Second image is pretraced, re-using what the first pretrace produced, and
> adding what is missing
> - Second image is rendered, no additional samples are needed
> - ...

In the current implementation, second and later calculations are not stored,
they just load the original calculations and calculate whatever is needed anew
without ever adding up and storing to the original radiosity file.  Even if
your 3rd frame of animation would benefit from them...

Post a reply to this message

"nemesis" <nam### [at] gmailcom> wrote:
> It seems to me faster results are dependent on parallelization of the
> "radiosity" calculation step and such step is being hampered of achieving such
> parallelism exactly by not being able to know in advance how much each thread
> should allow to "be visible".  At least in the case of different instances of
> povray rendering chunks of a larger image... as was the old approach to
> parallelism.  Not sure threads would improve the situation here, possibly yes,
> by having shared datastructures.

They *do* improve the situation. If I showed you a shot of what my current code
outputs, you couldn't tell it was shot in parallel - and it doesn't take
significantly more CPU time than a single thread working all on its own. CPU
time, mind you - not real time.

> OTOH, I'm aware of the locking issues...

What locking issues? There are no locking issues :) They're all sorted out.
Actually haven't been there in beta.29 at all (although the POV team hadn't
found the time yet to verify that), except for a tiny bit of more speed I was
able to gain.

> In the current implementation, second and later calculations are not stored,
> they just load the original calculations and calculate whatever is needed anew
> without ever adding up and storing to the original radiosity file.  Even if
> your 3rd frame of animation would benefit from them...

Heh - you ever tried? I must confess I didn't myself, but from what I see in the
code, using both "load_file" and "save_file" in the radiosity block should do
the trick. specifying "save_file" just causes the whole radiosity tree to be
dumped to file after rendering a frame, and doesn't care whether the samples
were loaded from file already or added during this frame.

(Unfortunately the current beta can't load/save right now)

Post a reply to this message

nemesis <nam### [at] gmailcom> wrote:
> It seems this feature, while a nice optimization trick, is what prevents one
> from reusing a full saved rad file to render a file from a different angle
> without taking any more samples and probably also the root cause of not being
> able to simply break an image into sections, render them separately and simply
> merge the final renders into a single image later.  Each instance do not know
> they should be taking samples from areas not seen in their particular view.

  I don't see how viewpoint-independence would help solve the problem. The
only thing a viewpoint-independent solution would do is calculate more
samples (including many samples which do not contribute anything to the
rendered image) and that's it. It might allow re-rendering the scene with
a moved camera without having to calculate more samples, but I don't see
how it would help any of the actual problems.

  A viewpoint-independent version of the current algorithm would (most
probably) not reduce lighting artifacts, make it any easier to get
good-looking lighting without tons of tweaking, and most relevantly,
would not make it any easier to distribute the calculation of the samples
at render-time. You would still have all the problems of the current
algorithm (eg. how to make one thread see the results of another, how
to avoid two threads calculating the same sample, etc).

> Perhaps this optimization was powerful then, but not quite as fitting in an era
> of multicore processors and parallel processing.

  The global illumination algorithm used by POV-Ray was slow then, and it's
slow nowadays. Deliberately making it even slower is not the way to go.

-- 
                                                          - Warp

Post a reply to this message

"clipka" <nomail@nomail> wrote:
> "nemesis" <nam### [at] gmailcom> wrote:
> > OTOH, I'm aware of the locking issues...
>
> What locking issues? There are no locking issues :) They're all sorted out.

Hmm, but you guys had quite some fun a while back with this discussion. ;)

> > In the current implementation, second and later calculations are not stored,
> > they just load the original calculations and calculate whatever is needed anew
> > without ever adding up and storing to the original radiosity file.  Even if
> > your 3rd frame of animation would benefit from them...
>
> Heh - you ever tried? I must confess I didn't myself, but from what I see in the
> code, using both "load_file" and "save_file" in the radiosity block should do
> the trick. specifying "save_file" just causes the whole radiosity tree to be
> dumped to file after rendering a frame, and doesn't care whether the samples
> were loaded from file already or added during this frame.

Aaaahhhhhhhhhhhhhhhh!!!

One of those magic trickery people shout about pov's rad! :P

Post a reply to this message

"nemesis" <nam### [at] gmailcom> wrote:

> Aaaahhhhhhhhhhhhhhhh!!!
>
> One of those magic trickery people shout about pov's rad! :P

So... is this an "Aaaahhhhhhhhhhhhhhhh!!! it doesn't work", or
"Aaaahhhhhhhhhhhhhhhh!!! If only I'd known that earlier" ?

Post a reply to this message