POV-Ray : Newsgroups : povray.off-topic : kd-trees rule! Server Time
9 Oct 2024 16:14:55 EDT (-0400)
  kd-trees rule! (Message 3 to 12 of 32)  
<<< Previous 2 Messages Goto Latest 10 Messages Next 10 Messages >>>
From: scott
Subject: Re: kd-trees rule!
Date: 13 Feb 2009 02:58:08
Message: <49952810@news.povray.org>
> I have implemented simple kd-tree support for my renderer ssRay and it
> already rocks! See the image attached. Now I can finally use more
> interesting models and render complex scenes without unbearable
> performance hit.

Wow that's really cool that you have done this all yourself so quickly, at 
this rate you'll be ahead of POV in a few months :-)

> Now the kd-tree is build with idiotic spatial median subdivision (ie.
> simply divide each node in half). That is stupid and I'll implement a
> surface area heuristic cost model next. That should improve performance
> considerably. But still, all looks pretty good so far!

I guess the goal here is to have the minimum number of steps down the 
kd-tree on average over all rays.  My very naive instinct is that you should 
split the nodes so there are equal numbers of triangles on each side, but 
that's probably wrong if you don't take into account the size of each 
triangle.

> I love this path tracing stuff :D

Yes if you have the time it can produce beautiful results, I have been using 
MCPov almost exclusively recently for renders, I hope that path-tracing can 
make it into official POV at some point.

BTW I have two questions about your tracer:

1) Multi-threaded, this can't be harder than running multiple instances at 
once, but does it already utilise all cores or how hard will it be to 
implement that?

2) How do you describe the scenes at the moment, are they all hard coded or 
do you have some scripting system already?


Post a reply to this message

From: Severi Salminen
Subject: Re: kd-trees rule!
Date: 13 Feb 2009 03:22:14
Message: <49952db6@news.povray.org>
scott wrote:

> Wow that's really cool that you have done this all yourself so quickly,
> at this rate you'll be ahead of POV in a few months :-)

:D Well, maybe not :) POV has a totally different approach to ray
tracing problem so it is hard to compare. With path tracing it is SOOOO
much easier to get many light related phenomena correct. With POV the
devs had to work hard to get the desired result as many phenomena are
separately implemented. With ssRay I don't have to shoot photons
separately to get caustics, I don't have to implement radiosity to get
global illumination and all related indirect effects etc. I get them for
free. Who doesn't like that :)

> I guess the goal here is to have the minimum number of steps down the
> kd-tree on average over all rays.  My very naive instinct is that you
> should split the nodes so there are equal numbers of triangles on each
> side, but that's probably wrong if you don't take into account the size
> of each triangle.

That is intuitively a good solution but in reality it is a bad one :D
Actually you want to cut off big empty regions early. That makes tracing
and traversing the tree cheap. If you take "object median" as the split
point you get two expensive child nodes. That usually is not a good idea.

But there are some very good documents and books about kd-tree creation
so I don't have to re-invent the wheel completely.

Basically you assign a certain cost for ray-object intersection test.
And certain cost for traversing an interior node of the kd-tree. Then
you figure out all the possible splitting plane points (in all 3
directions) and use the cost model to minimize the kd-tree cost. The key
here is to use the surface area of a node to get a probability that a
random ray would actually go through that node.

There is also another very interesting acceleration structure called
BIH: bounding interval hierarchy. It looks quite simple and still very
efficient. I'll stick to ke-tree for now but will definitely try BIH at
some point.

> Yes if you have the time it can produce beautiful results, I have been
> using MCPov almost exclusively recently for renders, I hope that
> path-tracing can make it into official POV at some point.

I hope that too! Fidos has shown that POV can be turned to path tracer.
And POV has a very strong SDL and other infrastructure so it would be
nice to have alternative rendering engine for it.

I still think that most POV users prefer the most realistic outcome of
the renderings. And unbiased is the best way to go if that is the main
goal. I hope the devs will consider that with POV4.

> 1) Multi-threaded, this can't be harder than running multiple instances
> at once, but does it already utilise all cores or how hard will it be to
> implement that?

Yes, ssRay is already multithreaded and scales very well - basically
perfectly. I have 2 cores running at ~100% and ssRay handles as many
threads as the user wants. Now I assign different pixel rows for each
thread. That is actually a good approach. I don't need mutexes and the
load is still quite well balanced in almost all scenes.


> 2) How do you describe the scenes at the moment, are they all hard coded
> or do you have some scripting system already?

Hardcoded at the present. At some point I have to implement a parser but
that is not a fun thing to do and developing a good SDL takes time...


Post a reply to this message

From: Severi Salminen
Subject: Re: kd-trees rule!
Date: 13 Feb 2009 03:28:42
Message: <49952f3a$1@news.povray.org>
triple_r wrote:

> It's snakes on a *** plane!  I like it.  Really, it's no small feat to get this
> far.  What's after kd-trees?

Thanks!

Hmm. I really don't know where to start after the SAH cost model of
kd-tree is finished. Most likely I'll spend some time refining the code
and making it more polished etc.

But I'll have to think about the material interface a bit more. Now I
have simple reflections and refractions which are not physically correct
and look unrealistic. Real reflections are angle dependent. A glass ball
reflects more at low angle than when looking form surface normal
direction. This has to be implemented properly.

I want to have Fresnel (or some good approximation like Shlick's
approximation). We'll see :D


Post a reply to this message

From: scott
Subject: Re: kd-trees rule!
Date: 13 Feb 2009 03:36:50
Message: <49953122$1@news.povray.org>
> There is also another very interesting acceleration structure called
> BIH: bounding interval hierarchy. It looks quite simple and still very
> efficient. I'll stick to ke-tree for now but will definitely try BIH at
> some point.

BTW there is an interesting document I came across a while back that 
involved working on sets of rays rather than just individual rays, might be 
an interesting read.

ftp://download.intel.com/technology/computing/applications/download/mlrta.pdf

> I still think that most POV users prefer the most realistic outcome of
> the renderings.

Agreed, after all we're using raytracers because we prefer quality over 
speed, as computers get faster it's only natural that we look for higher 
quality methods to use.

> Now I assign different pixel rows for each
> thread. That is actually a good approach. I don't need mutexes and the
> load is still quite well balanced in almost all scenes.

Oh yeh I was going to ask about anti-aliasing, do you get this for free too 
with your algorithm - or do you need to specifically jitter each primary ray 
somehow?

> Hardcoded at the present. At some point I have to implement a parser but
> that is not a fun thing to do and developing a good SDL takes time...

Yes of course, always the boring bit getting the UI working well, much more 
fun to work on the actual algorithm!


Post a reply to this message

From: Darren New
Subject: Re: kd-trees rule!
Date: 13 Feb 2009 03:49:32
Message: <4995341c$1@news.povray.org>
Severi Salminen wrote:
> I want to have Fresnel (or some good approximation like Shlick's
> approximation). We'll see :D

I want to see it do polarization. ;-)

-- 
   Darren New, San Diego CA, USA (PST)
   "Ouch ouch ouch!"
   "What's wrong? Noodles too hot?"
   "No, I have Chopstick Tunnel Syndrome."


Post a reply to this message

From: Severi Salminen
Subject: Re: kd-trees rule!
Date: 13 Feb 2009 03:53:37
Message: <49953511$1@news.povray.org>
Darren New wrote:

> I want to see it do polarization. ;-)

NOOOOOOOOO! Anything but that :D At least I have wavelength dependent
rendering higher on the list to allow realistic dispersion. But it's not
THAT high on the list.

-- 
Severi Salminen
http://www.iki.fi/severi


Post a reply to this message

From: Darren New
Subject: Re: kd-trees rule!
Date: 13 Feb 2009 04:06:28
Message: <49953814$1@news.povray.org>
Severi Salminen wrote:
> NOOOOOOOOO! Anything but that 

OK, fluorescence, then!

-- 
   Darren New, San Diego CA, USA (PST)
   "Ouch ouch ouch!"
   "What's wrong? Noodles too hot?"
   "No, I have Chopstick Tunnel Syndrome."


Post a reply to this message

From: triple r
Subject: Re: kd-trees rule!
Date: 13 Feb 2009 04:15:00
Message: <web.4995387855bca3e6ef2b9ba40@news.povray.org>
Severi Salminen <sev### [at] NOTTHISsaunalahtifiinvalid> wrote:
> Hardcoded at the present. At some point I have to implement a parser but
> that is not a fun thing to do and developing a good SDL takes time...

I've always thought it would be neat to leave it as an API.  POV-Ray's SDL is
already similar to C, but you'd also get the benefits of being able to use
numerical libraries, object-oriented programming, and of course, there's SPEED.
 All without actually having to write or maintain a parser!  The renderer would
just be a static or dynamic library.  Of course you have to compile and link
over and over, but how bad is that, really?

#include <ssray.h>
int main( int argc, char** argv ) {

     Scene s (argc, argv);

     vec x (0.0, 0.0, 0.0);
     vec dx (1.0, 0.0, 0.0);

     Union u;

     for( int i=0; i<100; i++) {
          u.push(
               sphere( x, 1.0,
                   pigment( Red )
               )
          );
          x += dx;
     )

     s.push(u);

     s.Render();

     return 0;
}

Upon reflection, maybe the answer is, "Pretty bad."  I think it's still the
approach I'd take since I don't actually know how to write a proper parser.

 - Ricky


Post a reply to this message

From: Invisible
Subject: Re: kd-trees rule!
Date: 13 Feb 2009 04:22:37
Message: <49953bdd@news.povray.org>
>> Hardcoded at the present. At some point I have to implement a parser but
>> that is not a fun thing to do and developing a good SDL takes time...
> 
> Yes of course, always the boring bit getting the UI working well, much 
> more fun to work on the actual algorithm!

 From where I'm sitting, building a parser is quite easy. Still, 
designing the language it's supposed to parse is another matter - 
depending on whether you want it to be Turing-complete. ;-)


Post a reply to this message

From: Warp
Subject: Re: kd-trees rule!
Date: 13 Feb 2009 09:02:37
Message: <49957d7b@news.povray.org>
Kd-trees are indeed quite handy. They are eg. used in 3D games in
certain situations where triangles need to be rendered in a certain order.

  For example semi-transparent polygons have to be rendered from
farthest-from-the-camera to closest. This is because if you render them
in the wrong order, the result would not look correct (the blending of
overlapping polygons would be in the wrong order and thus the result
incorrect). Also the Z-buffer cannot be used for semi-transparent polygons.
(If you used the Z-buffer and you render the polygons in the wrong order,
some of the polygons would not be rendered at all, even though they should.)

  So assume that you have, for example 10000 blades of grass, each one with
a semi-transparent texture, and thus you need to render then from farthest
to closest, regardless of where the camera is. How to do this?

  You could, of course, try to sort the triangles at each frame. However,
not only would this be inefficient (after all, you only have about 16 ms
to do *everything* that is needed to render the next frame, and sorting
these polygons would certainly not be the only thing you have to do), but
there's a huge problem with trying to sort the triangles by depth: The
comparison function is far from trivial to implement. Determining which
of two triangles is in front of the other is a surprisingly complicated
task.

  In fact, there is no comparison function to sort triangles by depth in
the general case. This might sound like a surprising result, but it indeed
is so. Three triangles may occlude cyclically each other, in which case
they don't have a well-defined order by depth. In other words, triangle A
may partially occlude triangle B, which partially occludes triangle C,
which partially occludes triangle A. In terms of comparison, you basically
have the situation where A < B and B < C and C < A, in which case these
three elements have no well-defined order.

  With opaque triangles cyclical occlusion is not a problem because the
Z-buffer takes care of hidden surface removal on a per-pixel basis. However,
with semi-transparent triangles you can't use the Z-buffer (and get a
correct result).

  The only way to render correctly cyclically-occluding triangles is
to split at least one of the triangles into two parts and render these
two parts separately (in the proper order compared to the other triangles).
However, you really don't want to start splitting triangles on the fly in
a real-time rendering engine.

  It is possible to split triangles as a preprocessing step (even at the
level design stage) so that they can be ordered unambiguously. However,
using a sorting algorithm to sort the triangles at each frame is not
really something you want to do. Even if the triangles could be sorted
unambiguously by depth, the comparison is complicated and heavy, as well
as the entire sorting algorithm, which would be O(n log n).

  Step in the kd-tree.

  If you add all the semi-transparent triangles into a kd-tree, you will
be able to render the triangles from farthest to closest with a simple
O(n) traversal of the tree, regardless of where the camera is
with respect to the triangles. The comparison to be made at each node
is much simpler than the comparison which would need to be done for a
sorting algorithm.

  And that's not all. When adding triangles to the kd-tree, it's trivial
to detect cyclical occlusion situations, and perform the necessary triangle
splitting, as a preprocessing step. After this has been done, no more splits
are ever necessary, and everything can be rendered in a depth order
regardless of where the camera is (using a simple linear traversal of the
kd-tree).

  (Of course which triangles to split is not unambiguous, and algorithms
can be developed to minimize the number of triangles which need to be
split, but this is only related to the preprocessing. The rendering stays
the same.)

-- 
                                                          - Warp


Post a reply to this message

<<< Previous 2 Messages Goto Latest 10 Messages Next 10 Messages >>>

Copyright 2003-2023 Persistence of Vision Raytracer Pty. Ltd.