On 11/24/20 2:10 PM, Kenneth wrote:
> Radiosity in animation suffers from a kind of jitter or flicker problem, even
> with high-quality settings... a result of the random nature of the rad 'light
> patches' from frame to frame. Using a SAVED radiosity file, from an initial
> camera viewpoint, would seem to solve that issue, but it's only partially
> successful: As the camera moves-- and the scene's objects move as well-- the
> saved radiosity file is no longer adequate to produce an expected smooth effect.
> 
> I came up with an interesting solution to part of this problem, at least for
> POV-ray v3.8xx and maybe 3.7 too, which is more easily explained in an animation
> test.
> 
...
> 
Cool idea and demo! Not positive, but I think if you both loaded and 
saved the radiosity samples the "new" ones would get appended to the 
file by POV-Ray (no need to edit). If so, it should be you could run a 
sampling of the total frames where you let POV-Ray keep adding to the 
radiosity file. This would be an initial pass for the purposes of 
creating the "all-frames" sample file used then for the all frames 
renders. Your file size concerns, still a worry, but...

I've not been following the radiosity discussions closely, but there is 
since v3.7 a +HR or High_Reproducibility option. Have you tried this? As 
far as I know, it still works only with radiosity though its initial aim 
was broader.

IIRC, what it does is change the seed to the render block "number" so 
even with multiple threads (SMP) you get the same random numbers (and so 
radiosity ray samples - at least by ordering) in any given block. The 
render image size and render block size would need to stay constant too 
of course.

Another option which obviously helps with reproducibility is running 
with one thread.

Bill P.

Post a reply to this message

William F Pokorny <ano### [at] anonymousorg> wrote:
> On 11/24/20 2:10 PM, Kenneth wrote:

> D) This uses TWO initial saved radiosity files, one made at the camera's
> start position and one made at the end (with different file names, of
> course!). Those files can be opened and inspected in POV-ray-- they are
> just text/data files of some kind-- then I simply copied-and-pasted the
> complete contents of the 2nd file onto the end of the 1st file, which is
> then re-saved. This 'composite' rad file is then used for the animation.

> Cool idea and demo! Not positive, but I think if you both loaded and
> saved the radiosity samples the "new" ones would get appended to the
> file by POV-Ray (no need to edit). If so, it should be you could run a
> sampling of the total frames where you let POV-Ray keep adding to the
> radiosity file...

Yes, you're right; I didn't think to try that easy trick until after I posted
here.  I had assumed that the two file-appending schemes *might* produce
different results, but they are the same.

Taking that idea to its logical conclusion, the following scheme is a
nicely-efficient way to generate the 'composite' saved radiosity file from
multiple camera viewpoints (for 'overlapping' rad data). Say you want a 50-frame
animation where the camera moves like
  <-3 + 6*clock,5,0>.
To create the composite rad file, the *initial* position would be  <-3,5,0>--
and the initial .INI file-saving settings need to be:
Radiosity_File_Name="RAD data" (or whatever name)
Radiosity_To_File=on
(animation turned OFF temporarily)

Then, to create the *full* saved composite file (the required interim process):
Radiosity_File_Name="RAD data"
Radiosity_To_File=on (to append more data to it)
Radiosity_From_File=on
Final_Frame=50
Subset_Start_Frame=2 (because frame 1 already has  its data saved)
Frame_Step=4 (or some other value-- because it is probably not necessary to save
rad data for *every* animation frame/camera position)

Then, to render the FINAL animation-with-radiosity:
Radiosity_File_Name="RAD data"
Radiosity_From_File=on
Final_Frame=50
.... along with the radiosity block's pretrace_start and pretrace_end both
changed to 1.0.

From experiments I've done, this works quite well-- with almost no flickering
rad-light patches.

Here's something interesting, though: For typical still-image renders, others
here have mentioned that a small-size (faster) initial radiosity render can be
made, while saving that data file for a *larger* render. This scheme works
reasonably OK for still images more or less, although there is a kind of 'lack
of radiosity detail' from the smaller render, and the saved file size is
correspondingly small as well, with fewer data entries. But this small-to-large
render scheme doesn't work well for creating a composite *animation* rad file--
there are still too many 'dancing light patches' in the final larger-size
animation. The only way the composite-saved-file idea works is to pre-render the
frames at their final intended pixel size, a slow process (while saving the
frame-to-frame data of course.)
>
> I've not been following the radiosity discussions closely, but there is
> since v3.7 a +HR or High_Reproducibility option. Have you tried this? As
> far as I know, it still works only with radiosity though its initial aim
> was broader.

So far, I have not gotten to that point in my experiments ;-) And I've been
running tests all week!
>
> IIRC, what it does is change the seed to the render block "number" so
> even with multiple threads (SMP) you get the same random numbers (and so
> radiosity ray samples - at least by ordering) in any given block. The
> render image size and render block size would need to stay constant too
> of course.

That's an interesting suggestion about render block size, that I had not thought
about; I need to experiment with that as well.
>
> Another option which obviously helps with reproducibility is running
> with one thread.
>

From initial experiments I've done on my lowly Windows 7 computer (with its 2
threads or cores or whatever), changing POV-Ray's Works_Threads setting in my
..INI file from 2 to 1 does not *seem* to affect the animation results re: the
look or placement of the radiosity light patches. I should probably do some more
tests though.

Post a reply to this message

"Kenneth" <kdw### [at] gmailcom> wrote:
> William F Pokorny <ano### [at] anonymousorg> wrote:
> > On 11/24/20 2:10 PM, Kenneth wrote:
>
> > D) This uses TWO initial saved radiosity files, one made at the camera's
> > start position and one made at the end (with different file names, of
> > course!). Those files can be opened and inspected in POV-ray-- they are
> > just text/data files of some kind-- then I simply copied-and-pasted the
> > complete contents of the 2nd file onto the end of the 1st file, which is
> > then re-saved. This 'composite' rad file is then used for the animation.
>
> > Cool idea and demo! Not positive, but I think if you both loaded and
> > saved the radiosity samples the "new" ones would get appended to the
> > file by POV-Ray (no need to edit). If so, it should be you could run a
> > sampling of the total frames where you let POV-Ray keep adding to the
> > radiosity file...
>
> Yes, you're right; I didn't think to try that easy trick until after I posted
> here.  I had assumed that the two file-appending schemes *might* produce
> different results, but they are the same.
>
> Taking that idea to its logical conclusion, the following scheme is a
> nicely-efficient way to generate the 'composite' saved radiosity file from
> multiple camera viewpoints (for 'overlapping' rad data). Say you want a 50-frame
> animation where the camera moves like
>   <-3 + 6*clock,5,0>.
> To create the composite rad file, the *initial* position would be  <-3,5,0>--
> and the initial .INI file-saving settings need to be:
> Radiosity_File_Name="RAD data" (or whatever name)
> Radiosity_To_File=on
> (animation turned OFF temporarily)
>
> Then, to create the *full* saved composite file (the required interim process):
> Radiosity_File_Name="RAD data"
> Radiosity_To_File=on (to append more data to it)
> Radiosity_From_File=on
> Final_Frame=50
> Subset_Start_Frame=2 (because frame 1 already has  its data saved)
> Frame_Step=4 (or some other value-- because it is probably not necessary to save
> rad data for *every* animation frame/camera position)
>
> Then, to render the FINAL animation-with-radiosity:
> Radiosity_File_Name="RAD data"
> Radiosity_From_File=on
> Final_Frame=50
> .... along with the radiosity block's pretrace_start and pretrace_end both
> changed to 1.0.
>
> From experiments I've done, this works quite well-- with almost no flickering
> rad-light patches.
>
> Here's something interesting, though: For typical still-image renders, others
> here have mentioned that a small-size (faster) initial radiosity render can be
> made, while saving that data file for a *larger* render. This scheme works
> reasonably OK for still images more or less, although there is a kind of 'lack
> of radiosity detail' from the smaller render, and the saved file size is
> correspondingly small as well, with fewer data entries. But this small-to-large
> render scheme doesn't work well for creating a composite *animation* rad file--
> there are still too many 'dancing light patches' in the final larger-size
> animation. The only way the composite-saved-file idea works is to pre-render the
> frames at their final intended pixel size, a slow process (while saving the
> frame-to-frame data of course.)
> >
> > I've not been following the radiosity discussions closely, but there is
> > since v3.7 a +HR or High_Reproducibility option. Have you tried this? As
> > far as I know, it still works only with radiosity though its initial aim
> > was broader.
>
> So far, I have not gotten to that point in my experiments ;-) And I've been
> running tests all week!
> >
> > IIRC, what it does is change the seed to the render block "number" so
> > even with multiple threads (SMP) you get the same random numbers (and so
> > radiosity ray samples - at least by ordering) in any given block. The
> > render image size and render block size would need to stay constant too
> > of course.
>
> That's an interesting suggestion about render block size, that I had not thought
> about; I need to experiment with that as well.
> >
> > Another option which obviously helps with reproducibility is running
> > with one thread.
> >
>
> From initial experiments I've done on my lowly Windows 7 computer (with its 2
> threads or cores or whatever), changing POV-Ray's Works_Threads setting in my
> ..INI file from 2 to 1 does not *seem* to affect the animation results re: the
> look or placement of the radiosity light patches. I should probably do some more
> tests though.

Interesting tests, If I remember correctly, Mental ray used to have a way to do
this as well... or maybe it was just for geometry export, but some kind of
incremental feature for animation.


Anyway, to my rather profane user ears your idea sounds sane to keep initial
radiosity data and only compute necessary additions for each new frame. So
you're saying that this radiosity data file is human readable. If each sample is
clearly identifiable, and can be split on a per line basis then it's worth
trying some kind of script (even maybe a notepad++ macro if it could recognize
and operate on numbers?) to do the following:

-copy the first ("rad1") and second ("rad2") frame's radiosity data files
calling these copies say "radtemp1" "radtemp2"

-pick a (low) precision threshold (a small enough number of digits after the
decimal points)to truncate all numbers of "radtemp1" and "radtemp2" to this
precision, so that you can...

-...iterate while keeping counted index, through each line of truncated
"radtemp1", find any line that now has the same patch coordinates in "radtemp2"
store its index in a list.

-suppress all lines corresponding to these indices in the original "rad2"

-And finally merge "rad2" at the end of rad1" to get your incremented but more
optimized rad2

However I would not be surprised if POV was already supposed to do all that kind
of stuff or even better optimization internally?

Post a reply to this message

"Mr" <nomail@nomail> wrote:

>
> ...So you're saying that this radiosity data file is human readable. If each
>  sample is clearly identifiable, and can be split on a per line basis then
> it's worth trying some kind of script (even maybe a notepad++ macro if it
> could recognize and operate on numbers?) to do the following:
>
> ...find any line that now has the same patch coordinates in "radtemp2"
> store its index in a list.
>
> ...And finally merge "rad2" at the end of rad1" to get your incremented but more
> optimized rad2
>

It would indeed be useful to optimize the final saved data file, if possible; my
animation test for *today*, with a moving camera, has a saved 'composite'
radiosity file with 47000+ lines of data! Here's a sample of what it looks like,
opened in POV-ray...

C1 6.40944 0 10.2727 7ffe7f 0.5393 0.5634 0.6627 0.0673439 2.02194 13b2aa
C1 5.07783 0 9.9777 7ffe7f 0.2932 0.2820 0.3779 0.0626854 19990.9 94c3e8
C2 4.67263 2.28262 10.8619 fd7f6d 0.2772 0.0367 0.0002 0.123517 2.80274 c81887
C1 5.22191 0 9.92912 7ffe7f 0.3809 0.2062 0.1807 0.0629131 2.52627 63f2ae
C1 5.40516 0 9.95757 7ffe7f 0.5724 0.6442 0.7586 0.0635047 19991.6 90cde2
C1 5.59453 0 9.99461 7ffe7f 0.4496 0.4617 0.5039 0.0641466 1.52217 33bfce

......etc. etc. Each line starts with C1 or C2. (And there's no obvious 'order'
to
the entries.) I have no idea what the data actually means.

Unfortunately, not a single line of that data seems to repeat; I tried hard to
find at least one repetition, with no luck. So my guess is that, as the *camera*
moves during the animation, totally-new 'coordinates' (or whatever) are
generated, with somewhat overlapping 'rad light patches'. I haven't yet tried a
test where the camera is static; maybe there *would* be duplicate entries that
could be eliminated.

BTW: From what I can tell so far, this 'additive' rad-data scheme (for the final
saved file) adds new light patches that sometimes overlap previous ones,
sometimes not. In effect, it seems to be adding to radiosity's COUNT value. So
it might be possible to 'relax' that value-- like, changing COUNT 200 to some
lower value-- to still get the original expected visual result. I haven't yet
tested the idea though.

Post a reply to this message

On 12/4/20 9:06 AM, Kenneth wrote:
> "Mr" <nomail@nomail> wrote:
> 
...
> 
> C1 6.40944 0 10.2727 7ffe7f 0.5393 0.5634 0.6627 0.0673439 2.02194 13b2aa
> C1 5.07783 0 9.9777 7ffe7f 0.2932 0.2820 0.3779 0.0626854 19990.9 94c3e8
> C2 4.67263 2.28262 10.8619 fd7f6d 0.2772 0.0367 0.0002 0.123517 2.80274 c81887
> C1 5.22191 0 9.92912 7ffe7f 0.3809 0.2062 0.1807 0.0629131 2.52627 63f2ae
> C1 5.40516 0 9.95757 7ffe7f 0.5724 0.6442 0.7586 0.0635047 19991.6 90cde2
> C1 5.59453 0 9.99461 7ffe7f 0.4496 0.4617 0.5039 0.0641466 1.52217 33bfce
> 
> ......etc. etc. Each line starts with C1 or C2. (And there's no obvious 'order'
> to
> the entries.) I have no idea what the data actually means.
> 

I dug up the source code not too long ago and made these notes:

---
C seems to stand for color. (There are other unused classifications)
The number after the D is depth. I've only ever seen 1 or 2 - maybe if
recursion depth increased?

The next three float values are the location.

The text string in position 5 is the normal encoded for +-(2 byte 
value/254) it looked.

Positions 6,7 and 8 are the three color channel values at the point.

In field 9 we have the harmonic mean. The inverse of the mean of the 
inverses of distances to all samples used to set a point's "radiosity 
view." This read as a double (long float) but later used in calculations 
as a float.

In field 10 the nearest sample distance found is recorded.

Field 11, like the normal field, is a string representing 3 x,y,z values 
pointing, roughly in the direction where the nearest distance sample was 
taken.

---
The floats are all read/scanned as doubles, but the file representation 
seems to be float. The internal usage in calculations is sometimes 
(always?) cast to float. Expect this means not too much difference 
between binary file encoding (as doubles) and current text encoding. 
However, if internally stored as all doubles and all only used as floats 
- the internal representation could probably be better?

FWIW.

... Yeah, hacking the radiosity file for effects is not something I 
recall anyone doing. It could be interesting. Color shifts. By global 
locations increase or decrease intensities. Insertion of samples by 
interpolation from existing ones? Completely contrived radiosity files
with gradients or whatever...

Aside: I suspect, there are some similarities to the saved photons file, 
but I've not gotten to even a first look at it...

Bill P.

Post a reply to this message

William F Pokorny <ano### [at] anonymousorg> wrote:
> On 12/4/20 9:06 AM, Kenneth wrote:
> > "Mr" <nomail@nomail> wrote:
> >
> ...
> >
> > C1 6.40944 0 10.2727 7ffe7f 0.5393 0.5634 0.6627 0.0673439 2.02194 13b2aa
> > C1 5.07783 0 9.9777 7ffe7f 0.2932 0.2820 0.3779 0.0626854 19990.9 94c3e8
> > C2 4.67263 2.28262 10.8619 fd7f6d 0.2772 0.0367 0.0002 0.123517 2.80274 c81887
> > C1 5.22191 0 9.92912 7ffe7f 0.3809 0.2062 0.1807 0.0629131 2.52627 63f2ae
> > C1 5.40516 0 9.95757 7ffe7f 0.5724 0.6442 0.7586 0.0635047 19991.6 90cde2
> > C1 5.59453 0 9.99461 7ffe7f 0.4496 0.4617 0.5039 0.0641466 1.52217 33bfce
> >
> > ......etc. etc. Each line starts with C1 or C2. (And there's no obvious 'order'
> > to
> > the entries.) I have no idea what the data actually means.
> >
>
> I dug up the source code not too long ago and made these notes:
>
> ---
> C seems to stand for color. (There are other unused classifications)
> The number after the D is depth. I've only ever seen 1 or 2 - maybe if
> recursion depth increased?
>
> The next three float values are the location.
>
> The text string in position 5 is the normal encoded for +-(2 byte
> value/254) it looked.
>
> Positions 6,7 and 8 are the three color channel values at the point.
>
> In field 9 we have the harmonic mean. The inverse of the mean of the
> inverses of distances to all samples used to set a point's "radiosity
> view." This read as a double (long float) but later used in calculations
> as a float.
>
> In field 10 the nearest sample distance found is recorded.
>
> Field 11, like the normal field, is a string representing 3 x,y,z values
> pointing, roughly in the direction where the nearest distance sample was
> taken.
>
> ---
> The floats are all read/scanned as doubles, but the file representation
> seems to be float. The internal usage in calculations is sometimes
> (always?) cast to float. Expect this means not too much difference
> between binary file encoding (as doubles) and current text encoding.
> However, if internally stored as all doubles and all only used as floats
> - the internal representation could probably be better?
>
> FWIW.
>
> ... Yeah, hacking the radiosity file for effects is not something I
> recall anyone doing. It could be interesting. Color shifts. By global
> locations increase or decrease intensities. Insertion of samples by
> interpolation from existing ones? Completely contrived radiosity files
> with gradients or whatever...
>
> Aside: I suspect, there are some similarities to the saved photons file,
> but I've not gotten to even a first look at it...
>
> Bill P.

Why not, ! Radiosity filters ! with that some inbuilt effect could be done like
the recent tests done for paint effect by Kenneth with his circular lights...
Back on initial topic, This is axactly the kind of data I hoped: So yes, the
radiosity algorithm is supposed to do unique samples to avoid wasting
computation, that is precisely why I mentioned to truncate the precision : so
that the samples closest would thus falsely appear as duplicates but the lines
suppressed in the initial data would rather be considered merged by that
(truncation) threshold criteria. Does that naive approach make sense to someone
more experienced than me, or am I missing something?

Post a reply to this message

"Kenneth" <kdw### [at] gmailcom> wrote:
>
> It would indeed be useful to optimize the final saved data file, if possible...
>
> Unfortunately, not a single line of [the saved data] data seems to repeat;
> I tried hard to find at least one repetition, with no luck...
> I haven't yet tried a test where the camera is static; maybe there *would*
> be duplicate entries that could be eliminated.
>

In all my animation/radiosity tests so far, I've used a moving CAMERA (and no
moving scene objects) to create the composite rad-data file. That seemed a
logical way to proceed; but I just did a 'static' camera test-- which produced
some interesting results:

Saving/appending 3 identical radiosity renders, with identical rad settings,
seems to be enough to prevent any 'new' rad light patches from being created
when running the final animation (assuming pretrace_start and pretrace_end are
changed to 1.0 for the final run; and assuming that the camera remains static.)
I would call it a '3-pass' technique. There are still a *few* stray patches
created, but they are very hard to spot. Of course, this doesn't take into
account any moving OBJECTS in the scene; I haven't gotten that far yet ;-)

As for the data file itself and its possible duplicated entries: I decided to
render and save a *tiny* radiosity file-- 4X3 pixels(!)-- made with a static
camera, and using the 3-pass trick. The final saved file is a 'composite' of the
3 renders and their data. The first 'pass' produced 108 saved data lines; the
2nd added only 9 lines; the third, 9 more. Then I tried to find duplicated data
lines among the total, but again no luck. So it *seems* that POV-ray
automatically removes any redundant data from an appended radiosity file.

Post a reply to this message

William F Pokorny <ano### [at] anonymousorg> wrote:
>
> I dug up the source code not too long ago and made these notes:

Fascinating! Thanks for taking the time to do this, and for posting your info.
>
> ---
> C seems to stand for color. (There are other unused classifications)
> The number after the D is depth. I've only ever seen 1 or 2 - maybe if
> recursion depth increased?

Yes, you're right, it's recursion_limit. (In my tests so far, I had set it to
2.) I ran another test with somewhat high-quailty settings (to produce a LARGE
saved rad file), and cranked resursion_limit up to 20. Now I see data lines
starting with C3,C4 etc. Interestingly, they 'top out' at about C8-- with fewer
entries too-- which agrees with the documentation about higher recursion_limit
settings having less and less of a visual effect on the render.
>
> The next three float values are the location...
> ...
>
> In field 10 the nearest sample distance found is recorded.
>
> Field 11, like the normal field, is a string representing 3 x,y,z values
> pointing, roughly in the direction where the nearest distance sample was
> taken.

From my animation tests with a moving camera, I assume that the x,y,z
values/direction vectors have something to do with CAMERA's viewpoint as well
(whatever it happens to be.) That is, if I understand your description
correctly.
>
>
> Aside: I suspect, there are some similarities to the saved photons file,
> but I've not gotten to even a first look at it...
>

IMO, there does seem to be a similarity:

I did an odd radiosity test, using a sky dome photo for the rad emitter, a gray
ground plane, and a perfectly-relective sphere for a scene object...
    (ambient 0 emission 0 diffuse 0 reflection 1.0)
... except that I cranked up its reflection to 8.0-- just an arbitrarily HIGH
value. I also added a normal to it. The result is that the sphere actually
reflects the incoming radiosity light.. and produces crude 'radiosity caustics'
on the plane, no photons required! I didn't realize that radiosity light could
be reflected, without using photons in some way. It seems that the two methods
do operate in a somewhat similar(?) way.

Post a reply to this message

On 12/4/20 11:43 AM, Mr wrote:
> William F Pokorny <ano### [at] anonymousorg> wrote:
... the
> radiosity algorithm is supposed to do unique samples to avoid wasting
> computation, that is precisely why I mentioned to truncate the precision : so
> that the samples closest would thus falsely appear as duplicates but the lines
> suppressed in the initial data would rather be considered merged by that
> (truncation) threshold criteria. Does that naive approach make sense to someone
> more experienced than me, or am I missing something?
> 
Expect quite a few things in play. There is some initial sampling 
density relative the the scene depending on numerous things for any 
given frame. If we change one or more of the dependencies enough, 
additional samples will be shot. Of those "perhaps" some will be added 
to the saved file.

In the moving camera case I think it likely new, duplicate or very 
nearby lines (samples) don't get added after a point because the some 
criteria (error, distance, ?) kicks in on seeing local context for the 
existing samples - and any in hand, "duplicate," potential sample is 
tossed in the bit bucket - but, I'm guessing.

---
I wonder if anyone has tried creating a radiosity file at a resolution 
larger than the final frame/scene?

I'm not aware of a way to force a stop immediately after photon / 
radiosity sample file creation. Is there one?

It would be handy to avoid the cost of the final pass render, saving of 
other intermediate and output files when just creating the larger 
radiosity file for the actual final frame/scene renders.

Bill P.

Post a reply to this message

On 12/4/20 11:46 AM, Kenneth wrote:
> "Kenneth" <kdw### [at] gmailcom> wrote:
...
> 2nd added only 9 lines; the third, 9 more. Then I tried to find duplicated data
> lines among the total, but again no luck. So it *seems* that POV-ray
> automatically removes any redundant data from an appended radiosity file.
> 
See my response to Mr as to why redundant (or near redundant) samples 
might not be getting added to the file.

Bill P>

Post a reply to this message