I was wading through the POV-Ray source code and noticed to my little
surprise that mesh is not the only primitive which uses reference counting
to save memory (ie. if you #declare such primitive and then insantiate
it several times, the main data will be kept in memory only once).

  These primitives use reference counting at least for part of their data:

blob
heightfield
lathe
mesh
polygon
prism
sor

  Just for fun I also looked which primitives use transformation
optimizations (a transformation optimization means that if you for
example make a sphere and translate it, no transformation matrix
will be created, but the center of the sphere will just be moved).

  These primitives use transformation optimization (along with which
transformations they optimize):

bicubic_patch: All transformations are applied directly to the points
box: translate, scale
plane: translate, rotate, scale
quadric: All transformations are applied to the quadric itself
sphere: translate, rotate, uniform scale
sphere_sweep: translate, rotate, uniform scale
triangle: All transformations are applied directly to the points

-- 
#macro M(A,N,D,L)plane{-z,-9pigment{mandel L*9translate N color_map{[0rgb x]
[1rgb 9]}scale<D,D*3D>*1e3}rotate y*A*8}#end M(-3<1.206434.28623>70,7)M(
-1<.7438.1795>1,20)M(1<.77595.13699>30,20)M(3<.75923.07145>80,99)// - Warp -

Post a reply to this message

Warp wrote:
> 
>   These primitives use reference counting at least for part of their data:

Quite interesting -- I was not aware of that.  Out of curiosity does 
anyone know why all objects can't use references for multiple copies?  I 
understand that it's some detail oin the implementation of POV-Ray, but 
I don't think I've ever heard what it was.

Post a reply to this message

> Quite interesting -- I was not aware of that.  Out of curiosity does
> anyone know why all objects can't use references for multiple copies?  I
> understand that it's some detail oin the implementation of POV-Ray, but
> I don't think I've ever heard what it was.

I actually very recently attempted to make an "object reference" patch which
does just that. I had a sort of working implementation, but at that point I
began to more fully understand the limitations which make it very difficult.

What it comes down to is this: all transforms are handled ahead of time. For
instance, when you transform a union, that transformation is applied right
then to every object within the union, and to their textures, interiors,
etc. Later, when an intersection is calculated, there is one intersection
point. This is the point in 3D space where the lighting is calculated from,
the normal of the object is calculated from, the texture is evaluated at,
etc.

Now say you have a reference to an object which is defined at the origin,
and the reference is translated to x = 10. (Internally, the actual object is
still at the origin, but the "object reference" wrapper has the
transformation applied.) When a ray intersects this object, where should the
intersection point be? Should it be near x=0 (relative to the original
object) or near x=10 (relative to the reference object)?

If it's near x=0, then the lighting will be calculated from that point,
which produces incorrect lighting. Other calculations may also go wrong,
since the intersection point is not really where the intersection happened
in 3D space.

But if it's near x=10, then the texture will be evaluated at that point,
which is wrong since the texture should be evaluated near x=0. Also, the
normal has to be calculated relative to x=0. Other calculations may also go
wrong.

And this is just the difficulty that transformations bring up. What if
no_shadow is applied to the object reference? It can't be copied to the
actual object since that would affect other references to the same object.
So we need both information about the actual object (its texture, no_shadow
and other flags, etc) and about the reference (additional texture, no_shadow
and other flags, all which should override the actual object's settings).
But we can't keep track of both at once.

The only practical way to avoid these problems is to prevent the user from
doing anything at all with object references; this then makes them useless.

One possible solution would be to find all the places where data from an
intersection is looked at, and determine which intersection point should be
considered, or which texture should be evaluated, etc. But there are so many
places in the code where these things happen that it would be easy to miss a
few, and have a buggy patch. Assumptions are made all throughout the source
code that an intersection happens in one place; those assumptions can't
practically be changed without a rewrite.

If POV-Ray were written in C++ (I believe/hope this is one of the changes
planned for 4.0?), then this wouldn't be so hard to solve. The Intersection
object would have functions like getNormal(), evaluateLighting(),
evaluateTexture(), and those could be rewritten to handle the special case
of object references.

I may continue to look for solutions to this problem with the restraint that
only transformations may be applied to an object reference, but I don't have
high hopes for it.

(The reason that none of these problems apply to mesh data or other
referenced data is that the only areas of code that deal with those data are
the respective objects that use them. Change the implementation of mesh data
and you only have to change the mesh object methods, not the entire source
code.)

 - Slime
 [ http://www.slimeland.com/ ]

Post a reply to this message

Kevin Wampler <wam### [at] cswashingtonedu> wrote:
> Quite interesting -- I was not aware of that.  Out of curiosity does 
> anyone know why all objects can't use references for multiple copies?  I 
> understand that it's some detail oin the implementation of POV-Ray, but 
> I don't think I've ever heard what it was.

  With most primitives copying by reference (instead of making a deep copy)
wouldn't make sense because it would only consume more memory.
  For example a sphere has only a center and a radius as data (besides
everything common to all objects). If you want to copy that sphere the
most efficient way of doing that is to simply make another sphere with
the same center and radius.

  It's only with primitives which contain (potentially) lots of data which
benefit from reference-counting. Mesh is one such primitive because this
single object can contain even hundreds of megabytes of data. Certainly
copying by reference is enormously more efficient than making a deep copy.
  Blobs also use reference counting for their data, as one blob can
contain lots of components inside it. Lathe and sor reference-count the
spline data (ie when you make several instances of the same pre-declared
lathe or sor, the spline inside them is not copied, but they all point
to the same spline data).

  One primitive type which would benefit a lot from reference counting
would be the CSG objects becaue they also can potentially contain lots
of other objects inside them.

  Now, there are several reasons why CSG does not do this.
  Firstly, the internal structure of the POV-Ray core code makes it
currently quite difficult to do this. A serious redesign would be
necessary in order to support this (which is something planned for pov4).

  Secondly, transformation optimizations and reference counting are
mutually exclusive. Primitives which have transformation optimizations
could not reside inside a csg object which uses reference counting.
  The transformation optimizations minimize the need to create a
transformation matrix for the object. If an object has a transformation
matrix it's somewhat slower to trace: Each trace against the object
requires a matrix multiplication. If the object does not have a
transformation matrix then this multiplication can be skipped.

  In the case of CSG supporting reference counting would be as simple
as having a CSG-specific transformation matrix. When a CSG is traced,
the ray would be transformed with the matrix of the CSG and then traced
against the objects inside it. However, these objects can also be
CSG themselves, and if they have transformation matrices, additional
multiplications would have to be made for each one that has. In
addition, if any object inside these CSG had matrices, the ray would
have to be multiplied by those as well.
  If I'm not mistaken currently CSG does not cause additional matrix
multiplications to be done, even if you have CSG inside CSG inside CSG.

  So it's a question of deciding which is worse, additional memory usage
or perhaps a bit slower rendering.
  (And take into account that the transformation matrix itself takes
some meory, so unless the CSG is very big, it could take even more memory
than without one.)

  It's not a trivial issue.

-- 
plane{-x+y,-1pigment{bozo color_map{[0rgb x][1rgb x+y]}turbulence 1}}
sphere{0,2pigment{rgbt 1}interior{media{emission 1density{spherical
density_map{[0rgb 0][.5rgb<1,.5>][1rgb 1]}turbulence.9}}}scale
<1,1,3>hollow}text{ttf"timrom""Warp".1,0translate<-1,-.1,2>}//  - Warp -

Post a reply to this message

Warp wrote:
>   So it's a question of deciding which is worse, additional memory usage
> or perhaps a bit slower rendering.
>   (And take into account that the transformation matrix itself takes
> some meory, so unless the CSG is very big, it could take even more memory
> than without one.)

Thanks for the answer.

I was thinking primarily of their usefulness with large CSGs (in 
particular in natural scenes with lots of trees, blades of grass, etc). 
  I would also imagine that it would not (in a redesign, not necessarily 
in the current povray) be possible to detect if an object is #declared 
and instanced multiple times and to make an estimation about weather it 
would be better use reference counting or not.  Worst case there could 
be a flag that could be set within the scene file.

At any rate, I'm glad to hear that these issues are likely to be 
considered in the pov4 redesign.  The developers seem to be pretty 
fantastic, so I'll trust that they'll do a good job on this issue as 
well.  For now I'll just content myself with meshes or more RAM.

Post a reply to this message

Slime wrote:
> 
> I actually very recently attempted to make an "object reference" patch which
> does just that. I had a sort of working implementation, but at that point I
> began to more fully understand the limitations which make it very difficult.
> 

That makes sense -- my curiosity is now sated.

Post a reply to this message

news:4213c3e2@news.povray.org...
>   I was wading through the POV-Ray source code and noticed to my little
> surprise that mesh is not the only primitive which uses reference counting
> to save memory (ie. if you #declare such primitive and then insantiate
> it several times, the main data will be kept in memory only once).
>
Good to know!
thanks for the tip
Marc

Post a reply to this message