clipka wrote:

> The approach of subdividing an image, and feeding each part to a separate task,
> works with regular raytracing only because even if parts of the scene interact
> through reflection, there is no data cache being built and re-used - because
> the chances of two mirrors showing exactly the same points are minimal (let
> alone that even then the incident angles would be different) and so caching
> wouldn't be worth the pain anyway.

An alternative approach would be to simply give each thread its
own cache. The threads will redo work done by other threads but it
may actually be less costly than locking stuff. It will be simple
to implement and faster than single-threaded pretrace unless
building the cache takes 100% of pretrace time. It removes
race conditions at least for individual pixels.

The problem remains that the image may look different on each
render if the assignment of image blocks to threads is dynamic
(load-balanced). This could be fixed by using a static block
assignment pattern for radiosity scenes (not too terrible, it
is rather unlikely that one thread gets all the slow blocks).

Unfortunately, the resulting image would then still depend
on the number of threads which were actually used, so anyone
who whished to exactly reproduce a scene would have to use
the same thread count.

Post a reply to this message

Darren New <dne### [at] sanrrcom> wrote:
> It's interpreted and dynamically typed, so it's certainly not something you
> want to try to write *fast* distributed code in. Erlang is distributed for
> reliability, not speed.  (And, FWIW, it's known to suck at floating point
> ops, too.)

Interestingly though, someone actually wrote a 3D modeller in that language.

However, this explains why Wings3D goes down to its knees when handling larger
models... which is a pity, because the user interface is exceptionally good.

Post a reply to this message

clipka wrote:
> Darren New <dne### [at] sanrrcom> wrote:
>> It's interpreted and dynamically typed, so it's certainly not something you
>> want to try to write *fast* distributed code in. Erlang is distributed for
>> reliability, not speed.  (And, FWIW, it's known to suck at floating point
>> ops, too.)
> 
> Interestingly though, someone actually wrote a 3D modeller in that language.

Yep. And it's not distributed, either. It uses all one thread, too. Although 
there are features for recompiling it and restarting it without actually 
closing the model you're working on.

> However, this explains why Wings3D goes down to its knees when handling larger
> models... 

That and being "functional" (rather than functional without the quotes, like 
Haskell), meaning in Erlang you have to copy any data structure you want to 
change, without actually being functional enough to make it possible for the 
compiler to optimize based on immutability of variables.

-- 
   Darren New, San Diego CA, USA (PST)
   The NFL should go international. I'd pay to
   see the Detroit Lions vs the Roman Catholics.

Post a reply to this message

> -----Original Message-----
> From: clipka [mailto:nomail@nomail]
> Some serious feedback on this anyone, before it gets swamped in posts
> about
> mutexes?

Yeah.

From what I understand, POV does a pre-trace to precompute a sample
tree.  Then, during the actual, render, POV creates any additional
samples needed.

What if there were a way, during the pretrace, to predict where
additional samples would be needed?  If so, then all the samples could
be taken during the pretrace, with no additional sampling during the
final render, with no loss in quality.

...Ben Chambers
www.pacificwebguy.com

Post a reply to this message

"Chambers" <ben### [at] pacificwebguycom> wrote:
> From what I understand, POV does a pre-trace to precompute a sample
> tree.  Then, during the actual, render, POV creates any additional
> samples needed.
>
> What if there were a way, during the pretrace, to predict where
> additional samples would be needed?  If so, then all the samples could
> be taken during the pretrace, with no additional sampling during the
> final render, with no loss in quality.

Yes, this would solve the multitasking issues (talking about 100%
reproducibility of shots) during main render.

Unfortunately, I know of only two approaches to collect all the samples needed
before the main render:

(1) Just plain stupidly cover all the surfaces in the whole scene with samples -
which will be a problem for scenes that stretch into infinity.

(2) Perform the full render twice.

The problem is that unless you really *do* the final render, you never know
which regions of the shot may ultimately be reached by some stray ray.

To give a trivial example: If you turn on focal blur, you will find that the
final render will take more radiosity samples - especially in the very first
and very last lines. That's because regions normally just outside the visible
region now need to be mixed in.

Another example are small or strongly-curved reflecting details that pretrace
may not have encountered.

So if we want to really do a complete pretrace, we end up taking the full-detail
shot twice - once with radiosity sampling on, and once without further sampling.
And what's worse: The first pass will be the more expensive one. And it seems we
want to do it in a single task to avoid the trouble already mentioned.

And what on earth are we doing the final render for then?

I don't really think this gets us anywhere :)

Post a reply to this message

Christian Froeschlin <chr### [at] chrfrde> wrote:
> An alternative approach would be to simply give each thread its
> own cache. The threads will redo work done by other threads but it
> may actually be less costly than locking stuff. It will be simple
> to implement and faster than single-threaded pretrace unless
> building the cache takes 100% of pretrace time. It removes
> race conditions at least for individual pixels.

Yup, thought of it too.

> Unfortunately, the resulting image would then still depend
> on the number of threads which were actually used, so anyone
> who whished to exactly reproduce a scene would have to use
> the same thread count.

Not necessarily so. If the image is *always* divided into the same number of
blocks, *always* using separate trees for any two blocks even if they are
processed by the same thread, and *always* merging the trees in the same order,
then we do have 100% reproducibility regardless of thread count.

If in addition we have each thread render only one pretrace pass at a time, then
consolidate all collected samples into one tree before proceeding with the next
pretrace pass, we also reduce the number of surplus samples taken.

In the first pretrace pass, the probability is quite low that two tasks need
samples so close together that one of them would actually be not needed.

In the second pretrace pass however, the first pass will already have generated
quite a number of "deeper-bounce" samples to start with.


This brings me to some *GOOD* idea... hey, thanks for the inspiration!

(I'll go into detail in a separat thread... this one has accumulated too much
"spin-offs" for my taste.)

Post a reply to this message

clipka wrote:
> Some serious feedback on this anyone, before it gets swamped in posts about
> mutexes?

I think one of the best ways to start is to get the tree building and 
re-sampling working again in the single-render thread case. From there you 
can make progress towards multithreading.

	Thorsten

Post a reply to this message

Thorsten Froehlich <tho### [at] trfde> wrote:
> I think one of the best ways to start is to get the tree building and
> re-sampling working again in the single-render thread case. From there you
> can make progress towards multithreading.

I think it doesn't make sense to put any work into POV's radiosity without
having at least some rough idea how it will be operating.

And making a POV-version that will support SMP only for non-radiosity scenes
isn't worth the hassle in my eyes: To get my quad-cores working simultaneously
on a non-radiosity scene, I can always start four full-fledged render processes
to crunch away on the same scene, forcing them to produce different jittering
and merging the results later - so I can get away with less antialias / focal
blur samples and lower area light resolution to get the whole thing speed up.


Ah, speaking of it: 100% reproducibility seems a holy cow in POV, but there are
situations where one might deliberately want variations in jitter effects from
shot to shot. So it would be nice to have a command line / ini parameter that
basically says "add X to all jitter-related random generator seeds" (where X
may even be something derived from current system time).

Setting a different antialias jitter value helps in practice, but I think it's
not the cleanest approach.

Post a reply to this message

clipka wrote:
> Thorsten Froehlich <tho### [at] trfde> wrote:
>> I think one of the best ways to start is to get the tree building and
>> re-sampling working again in the single-render thread case. From there you
>> can make progress towards multithreading.
> 
> I think it doesn't make sense to put any work into POV's radiosity without
> having at least some rough idea how it will be operating.

Well, given the code mostly works, it is just a matter of fixing a few bugs 
and then having a working reference implementation does make quite a bit of 
sense...

	Thorsten

Post a reply to this message

Thorsten Froehlich <tho### [at] trfde> wrote:
> Well, given the code mostly works, it is just a matter of fixing a few bugs
> and then having a working reference implementation does make quite a bit of
> sense...

As it seems to me, the problem with the code, as it is now, is that it was
attempted to change too many things at once:

- Migrating from C to C++
- Introducing Multithreading
- Introducing some optimizations
- Changing some features
- Doing all of this at different paces for different components

In addition, the refactoring approach wasn't ideal. "TODO FIXME" is never a good
solution to problems that arise during refactoring, because you effectively
break the code and have to re-assemble it later. So you have a time in between
when you cannot do any regression testing on the "amputated" code, and when you
are able to do it again, the code may have changed too much overall to do any
sensible regression testing.

In the case of POV 3.7.0 beta, some parts of the radiosity code have been
amputated to ease the transition from C code (global data) to C++ code
(object-internal data storage). If that would have been the only changes for
now, things would be much easier for me: I could be sure that the beta must
produce exactly the same output as the 3.6.1c, up to the very last pixel.

Now I don't know: I get different output, but is it really wrong?. After all, as
a first step towards multithreading the picture is now pretraced in a different
order (block by block, instead of line by line), which changes the choice of
samples taken; even if I would change the processing order back, the main
render would still be done block by block now. I can't possibly change *all*
the code back to 3.6.1c.

Then there is the Gamma thing: The beta.29 seems to produce a higher contrast in
radiosity shots, but is that a radiosity bug, or do I see some effect of the
Gamma stuff here?

In addition, I find code that seems to do things differently than the 3.6.1 -
not on an overall design level, but in small details. Some are obviously
intended as features - but they reduce my ability even further to assess
whether the output I get is good or bad.

Of course, last not least there is the problem that 3.6.1c radiosity was bad as
well. I actually think that what I currently have produces significantly better
results not only compared to beta.29, but also to 3.6.1c.

The most perplexing thing is that I get these results at very low costs (for
reasons I don't even understand; at the moment I'm just skimming through the
radiosity code, leveling out everything that seems to have changed from 3.6.1c
and beta.29).

For example, a 640x512 pixel version of my radiosity-based "Claustrophobia" shot
for the TINA-CHeP competitiontakes 7830 seconds of CPU time with the beta.29 on
my machine, but just 2370 seconds with my modifications - less than a third of
the time. Both versions were running at full throttle. Both produced images
that seem to differ only in random details.

(see http://www.tc-rtc.co.uk/imagenewdisplay/stills/index151.html what kind of
scene I'm talking about)

The 3.6.1c still hasn't finished more than 60% of the shot after ~8000 seconds
of CPU time; I'm curious how the results will compare.


You mention that it's "just a matter of fixing a few bugs" - if that is so, it
would be helpful to get some description of them.

Post a reply to this message