Fun fact:

Trying to read obj files directly turns out to be only marginally faster
than parsing mesh2 data (about 3% as it seems).

Post a reply to this message

> Fun fact:
>
> Trying to read obj files directly turns out to be only marginally faster
> than parsing mesh2 data (about 3% as it seems).

I thought obj files had an ascii and binary version, but I was wrong. 
That seems to be stl files. Maybe the binary stl format is faster to read?

https://en.wikipedia.org/wiki/STL_(file_format)

Post a reply to this message

Le 14/09/2016 à 08:39, scott a écrit :
>> Fun fact:
>>
>> Trying to read obj files directly turns out to be only marginally faster
>> than parsing mesh2 data (about 3% as it seems).
>
> I thought obj files had an ascii and binary version, but I was wrong.
> That seems to be stl files. Maybe the binary stl format is faster to read?
>
> https://en.wikipedia.org/wiki/STL_(file_format)
>
You can give it a try in hgpovray (there is a stl_load and stl_save )

https://github.com/LeForgeron/povray/wiki/Tesselation

The main issue with povray mesh import is the unique indexing of 
vectors: when the data structure is not like mesh2, each new component 
need to perform a search in previous values.

Even the parsing of a mesh get slowed down as the number of triangles 
increased. Fast for 100, Normal when reaching 10 000, crawling after 1 
000 000.

Mesh2 avoid the search because the uniqueness is done before (by the 
generator of mesh2).

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> Fun fact:
>
> Trying to read obj files directly turns out to be only marginally faster
> than parsing mesh2 data (about 3% as it seems).

Interesting! Too bad it's not alembic files which, as a caching format, will
probably benefit from a simpler/faster export flow...
Still OBJ is interesting! do you think it would be worth exporting Blender mesh
geometry to OBJ instead of Mesh2 or rather let you investigate and optimize the
mesh2, so that it will then be even faster ! :-) ?

Post a reply to this message

Am 14.09.2016 um 08:39 schrieb scott:
>> Fun fact:
>>
>> Trying to read obj files directly turns out to be only marginally faster
>> than parsing mesh2 data (about 3% as it seems).
> 
> I thought obj files had an ascii and binary version, but I was wrong.
> That seems to be stl files. Maybe the binary stl format is faster to read?
> 
> https://en.wikipedia.org/wiki/STL_(file_format)

Faster to read maybe, but severely limited in functionality: The
official specification provides for nothing more than flat untextured
geometry.

Also, my guess would be that STL export is slightly less widespread than
OBJ export.


Besides, there are other things that bog down import; for instance, on
my system an import of two Gen.3 DAZ figures with hair takes about 11
seconds to load the image maps (and those are only the diffuse maps; the
exporter didn't even include specular maps or normal maps), some 3-4
seconds to actually load the mesh data, and another whopping 9 seconds
to build the internal bounding hierarchy (which can be suppressed with
"hierarchy off", but you absolutely positively don't want to do that,
/especially/ for big meshes)

Post a reply to this message

Am 14.09.2016 um 09:46 schrieb Le_Forgeron:

>> I thought obj files had an ascii and binary version, but I was wrong.
>> That seems to be stl files. Maybe the binary stl format is faster to
>> read?
>>
>> https://en.wikipedia.org/wiki/STL_(file_format)
>>
> You can give it a try in hgpovray (there is a stl_load and stl_save )
> 
> https://github.com/LeForgeron/povray/wiki/Tesselation
> 
> The main issue with povray mesh import is the unique indexing of
> vectors: when the data structure is not like mesh2, each new component
> need to perform a search in previous values.

Ah, yes, and then there's that -- I totally forgot about that in
conjunction with STL. So OBJ will be faster anyway.

But I would put it the other way round: That's the main issue of naive
file formats which store triangles as a list of vertex coordinates,
rather than having a list of vertices and only storing vertex indices
per triangle.

That's absolutely not unique to POV-Ray; every sane piece of software
handling meshes does that internally. So exporting to a naive format and
re-importing is just a plain waste of time.

> Even the parsing of a mesh get slowed down as the number of triangles
> increased. Fast for 100, Normal when reaching 10 000, crawling after 1
> 000 000.

I guess some improvements could be made there. AFAIK the mesh code uses
a homebrew hash map for this purpose; large meshes may easily outgrow
the hash table, and a search tree would be more performant there.

But if speed is really of paramount importance, and memory not so much,
there's always the possibility of just storing duplicate vertices.

> Mesh2 avoid the search because the uniqueness is done before (by the
> generator of mesh2).

Not even by the generator of mesh2; the uniqueness is inherent in every
mesh editor that deserves the name (otherwise it would be a disconnected
triangle editor).

Post a reply to this message

Am 14.09.2016 um 10:08 schrieb Mr:
> clipka <ano### [at] anonymousorg> wrote:
>> Fun fact:
>>
>> Trying to read obj files directly turns out to be only marginally faster
>> than parsing mesh2 data (about 3% as it seems).
> 
> Interesting! Too bad it's not alembic files which, as a caching format, will
> probably benefit from a simpler/faster export flow...

Uh, I'd rather not. With Alembic being a quite modern format, users may
expect support for a few more features than just polygon meshes, such as
NURBS and things, and supporting those would be a _lot_ of work. OBJ, on
the other hand, is old enough that people know better than to expect
such bells and whistles to work with all software.

Also, I haven't yet found /any/ documentation whatsoever about the file
format itself, only a ready-made library; so implementing even a tiny
bit of it would require more than the mere ~500 lines of code I added
for OBJ import; it may also suffer in speed from additional code
required to convert the data structures provided by the library to those
used by POV-Ray internally.

> Still OBJ is interesting! do you think it would be worth exporting Blender mesh
> geometry to OBJ instead of Mesh2 or rather let you investigate and optimize the
> mesh2, so that it will then be even faster ! :-) ?

When it comes to import speed, I would suspect OBJ to maintain a slight
advantage over mesh2, because the formats are reasonably similar in
structure, with OBJ being just that little bit less verbose(*). Mesh2
parsing /per se/ cannot be reasonably sped up any further as it uses
POV-Ray's general parser infrastructure (by contrast, OBJ import
accesses the file directly; which also means that you can't generate OBJ
data "on the fly" procedurally, like you can with mesh2 data, except by
the tedious process of writing a temporary file). Any potential for
speedup of mesh2 parsing is in pieces of code that OBJ import would also
benefit from.


(* Or at least it has the potential to be, if exporters group the
triangles by material before writing them. If material changes
frequently between consecutive triangles in the OBJ file, the format can
get /very/ verbose.)

Post a reply to this message

clipka <ano### [at] anonymousorg> wrote:
> Am 14.09.2016 um 10:08 schrieb Mr:
> > clipka <ano### [at] anonymousorg> wrote:
> >> Fun fact:
> >>
> >> Trying to read obj files directly turns out to be only marginally faster
> >> than parsing mesh2 data (about 3% as it seems).
> >
> > Interesting! Too bad it's not alembic files which, as a caching format, will
> > probably benefit from a simpler/faster export flow...
>
> Uh, I'd rather not. With Alembic being a quite modern format, users may
> expect support for a few more features than just polygon meshes, such as
> NURBS and things, and supporting those would be a _lot_ of work. OBJ, on
> the other hand, is old enough that people know better than to expect
> such bells and whistles to work with all software.
>
> Also, I haven't yet found /any/ documentation whatsoever about the file
> format itself, only a ready-made library; so implementing even a tiny
> bit of it would require more than the mere ~500 lines of code I added
> for OBJ import; it may also suffer in speed from additional code
> required to convert the data structures provided by the library to those
> used by POV-Ray internally.
>
> > Still OBJ is interesting! do you think it would be worth exporting Blender mesh
> > geometry to OBJ instead of Mesh2 or rather let you investigate and optimize the
> > mesh2, so that it will then be even faster ! :-) ?
>
> When it comes to import speed, I would suspect OBJ to maintain a slight
> advantage over mesh2, because the formats are reasonably similar in
> structure, with OBJ being just that little bit less verbose(*). Mesh2
> parsing /per se/ cannot be reasonably sped up any further as it uses
> POV-Ray's general parser infrastructure (by contrast, OBJ import
> accesses the file directly; which also means that you can't generate OBJ
> data "on the fly" procedurally, like you can with mesh2 data, except by
> the tedious process of writing a temporary file). Any potential for
> speedup of mesh2 parsing is in pieces of code that OBJ import would also
> benefit from.
>
>
> (* Or at least it has the potential to be, if exporters group the
> triangles by material before writing them. If material changes
> frequently between consecutive triangles in the OBJ file, the format can
> get /very/ verbose.)

So there is not a single feature in mesh2 that we'd lose when switching to OBJ?
could we still use smoothing groups? no limit in the number of points/faces...?
Why did you choose it over some binary format like 3DS ? wouldn't this be even
faster?  maybe the partial update that could one day be implemented for
animations?

Post a reply to this message

"Mr" <nomail@nomail> wrote:

>
> So there is not a single feature in mesh2 that we'd lose when switching to OBJ?
> could we still use smoothing groups? no limit in the number of points/faces...?
> Why did you choose it over some binary format like 3DS ? wouldn't this be even
> faster?  maybe the partial update that could one day be implemented for
> animations?
I also forgot the current hack we use for vertex colors using pov pigments per
vertex, will it still be possible?

Post a reply to this message

Am 14.09.2016 um 11:45 schrieb Mr:

> So there is not a single feature in mesh2 that we'd lose when switching to OBJ?
> could we still use smoothing groups? no limit in the number of points/faces...?

mesh2 does not support smoothing groups; if the exporting program uses
them, it has to convert them to vertex normals (which are a much more
flexible mechanism) for mesh2 export; same deal for OBJ, so no loss there.

OBJ does not have an intrinsic limit to the number of vertices or faces;
exporters and imoporters may, and in fact there is /some/ upper limit
intrinsic in the new OBJ import code, but those limits are the same as
for the mesh2 parser, so no loss there either.


> Why did you choose it over some binary format like 3DS ? wouldn't this be even
> faster?  maybe the partial update that could one day be implemented for
> animations?

Compatibility and simplicity. OBJ is probably /the/ one and only
baseline mesh interchange format that /all/ mesh-based 3D tools support,
has a very straightforward structure, and besides is a text-based format
and thus can be read in an editor, which is an advantage in debugging.

3DS, on the other hand, has a rather complex nested structure, and also
has some other drawbacks; coincidently, those happen to include the very
topics you mentioned earlier:

- The format does not support explicit vertex normals, making it
necessary to fall back to smoothing groups instead; importing those into
POV-Ray would be a tremendous hassle, as POV-Ray internally operates
with vertex normals, which would have to be generated from the smoothing
groups, which in turn requires a costly look at neighboring triangles,
and may cost you more processing time than the binary import would save
you. Also, if the exporting program natively operates with vertex
normals as well, or uses a different algorithm to compute vertex normals
from smoothing groups for rendering, smoothing information will suffer
in the process.

- The format is limited to 65536 vertices and triangles per mesh.

Plus (though this would only be relevant when adding texture import):

- The format limits texture file names to 8.3 format (and without path
information I presume), requiring the exporter to copy the textures around.

Post a reply to this message