Hi(gh)!

Still aiming at POVEarth/POVghanistan (and also high-quality renderings 
of other Solar System bodies) - but currently, there is a bottleneck 
when it comes to generating mesh2s from 16-bit (or whatever format) 
heightfields!

The Japanese ASTER mission elevation tiles have a resolution of 3601 by 
3601 measuring points (i. e. pixels in a heightfield generated from 
these data) - but even with 16 GiB of RAM, I had to decrease the 
resolution to 2600 by 2600, otherwise POV-Ray would crash.

Of course this is because POV-Ray generates full ASCII code rather than 
(like with heightfields) binary data to internally represent the mesh2 
object.

Have there ever been any attempts to write a patch to make binary meshes 
feasible with POV-Ray?

See you in Khyberspace!

Yadgar

Post a reply to this message

=?UTF-8?B?SsO2cmcgJ1lhZGdhcicgQmxlaW1hbm4=?= <yazdegird@gmx.de> wrote:
> Hi(gh)!
>
> Still aiming at POVEarth/POVghanistan (and also high-quality renderings
> of other Solar System bodies) - but currently, there is a bottleneck
> when it comes to generating mesh2s from 16-bit (or whatever format)
> heightfields!
>
> The Japanese ASTER mission elevation tiles have a resolution of 3601 by
> 3601 measuring points (i. e. pixels in a heightfield generated from
> these data) - but even with 16 GiB of RAM, I had to decrease the
> resolution to 2600 by 2600, otherwise POV-Ray would crash.
>
> Of course this is because POV-Ray generates full ASCII code rather than
> (like with heightfields) binary data to internally represent the mesh2
> object.
>
> Have there ever been any attempts to write a patch to make binary meshes
> feasible with POV-Ray?
>
> See you in Khyberspace!
>
> Yadgar

what's the format for the ASTER tiles?  it should be fairly simple to write a
small program to decompose the data into a series of mesh2 objects and
then create a union/merge of the individual objects.

Regards,
A.D.B.

Post a reply to this message

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Le 15/06/2015 23:41, Anthony D. Baye a écrit :
> =?UTF-8?B?SsO2cmcgJ1lhZGdhcicgQmxlaW1hbm4=?= <yazdegird@gmx.de>
> wrote:
>> Hi(gh)!
>> 
>> Still aiming at POVEarth/POVghanistan (and also high-quality
>> renderings of other Solar System bodies) - but currently, there
>> is a bottleneck when it comes to generating mesh2s from 16-bit
>> (or whatever format) heightfields!
>> 
>> The Japanese ASTER mission elevation tiles have a resolution of
>> 3601 by 3601 measuring points (i. e. pixels in a heightfield
>> generated from these data) - but even with 16 GiB of RAM, I had
>> to decrease the resolution to 2600 by 2600, otherwise POV-Ray
>> would crash.
>> 
>> Of course this is because POV-Ray generates full ASCII code
>> rather than (like with heightfields) binary data to internally
>> represent the mesh2 object.
>> 
>> Have there ever been any attempts to write a patch to make binary
>> meshes feasible with POV-Ray?
>> 
>> See you in Khyberspace!
>> 
>> Yadgar
> 
> what's the format for the ASTER tiles?  it should be fairly simple
> to write a small program to decompose the data into a series of
> mesh2 objects and then create a union/merge of the individual
> objects.
> 

Aster elevation image (there is a Quality image too, but it's
irrelevant for ) is a tiff (raster) of signed 16 bits value (at one
meter for each value, 0 is sea level, reference is the WGS84 geoid
(but we can have it like a sphere to ease our life), -9999 is a
special "void" pixel value).

Each tile is 3601 x 3601 pixels (of signed 16 bits), each corner is an
integer longitude and latitude, and it covers 1 x 1 degree.

(the border of each tile is duplicated on the neighbouring tiles, the
overlap is only 1 pixel wide)

and now a bit of maths for a whole planet (despite Aster tiles being
available only for land-mass and only between +83° and -83° latitudes):

3601 pixels x 3601 pixels x 2 bytes x 180 latitudes x 360 longitudes
need 1565 Gigabytes.

That's still a bit of memory, so POVEarth is an unreachable dreams for
still some long years (at least 15, time for the usual computer to
have 2 tera of Ram).

POVghanistan : south is 29°, west 60°, east 75°, north 39°, that would
be about 150 tiles , already 3.7 Giga of compact data. But
transphorming them in mesh is a bad idea.
Each coordinates/elevation ( 2 bytes in compact format ) is going to
be a unique 3D coordinates (3 singles, about 12 bytes), and 1 944 000
000 ( numbre of points) is nearly half the space of 32 bits index.
Do not ask why the MeshIndex type is a signed int, but this reduces to
31 bits the maximal number of vertices and triangles.

And now come the troubles: each triangle must reference a normal and 3
vertices. There is approximately 2 triangles per vertex, so there is 3
888 000 000 triangles, far too much unless the code is changed to use
a signed 64 bits as MeshIndex.
Yet that's only half of the problem: the memory footprint of each
triangle is enormous (it has to handle smooth triangle, uv mapping and
texture interpolation, options that might be irrelevant for
POVghanistan, yet they consumes many bytes per triangle): 69 bytes (if
I counted right) with 32 bits indexes, 121 bytes with 64 bits indexes.

That's a lot of memory... yet the vector of the mesh are only single
precision (so down to 7th number), which might be not enough for
something like POVghanistan. Upgrading to double precision also double
the memory for vertices' storage.

It might be simpler (sort of) to have an object that behave like a
sphere with a sea-level-radius and a factor for elevation provided in
signed-16-bit tiff images at integer longitudes and latitudes: each
tiff being a sheared (transformation matrix) height-field (rotated to
match the longitude & latitude on the sphere).
I'm not sure the compensation of the roundness of earth is necessary
on each 1°x1° tile, but it could be done later as a post-process when
generating each height-fields if really needed. (it cannot be done
with a transform)

Something like:

foobar{ 6371, 0.001
  tile { 30, 70, "ASTGTM2_N30E070_dem.tif" }
  tile { 31, 70, "ASTGTM2_N31E070_dem.tif" }
  tile { 32, 70, "ASTGTM2_N32E070_dem.tif" }
}

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iJwEAQEIAAYFAlWBLGEACgkQhKAm8mTpkW22YAP/R4JwmClK8g2f0cR71g3Sl2vF
/C1q1t5qFmsEDTC2nCSI2Ig2CCQOXM7L0n13xWTcO7s+JRiORwNOk6Uk3KddGUyf
geuF+AXWQpj+f2v/XpAOgZs7F2VBypSbVmkSPzYDiKEyU/yZCH74lkFERpygDt7Q
Z3hCRhvNgoAtSLtJ40E=
=aL6/
-----END PGP SIGNATURE-----

Post a reply to this message

That is a no, then?  ;-)


On 17/06/2015 09:14, Le_Forgeron wrote:
>
>
> Aster elevation image (there is a Quality image too, but it's
> irrelevant for ) is a tiff (raster) of signed 16 bits value (at one
> meter for each value, 0 is sea level, reference is the WGS84 geoid
> (but we can have it like a sphere to ease our life), -9999 is a
> special "void" pixel value).
>
> Each tile is 3601 x 3601 pixels (of signed 16 bits), each corner is an
> integer longitude and latitude, and it covers 1 x 1 degree.
>
> (the border of each tile is duplicated on the neighbouring tiles, the
> overlap is only 1 pixel wide)
>
> and now a bit of maths for a whole planet (despite Aster tiles being
> available only for land-mass and only between +83° and -83° latitudes):
>
> 3601 pixels x 3601 pixels x 2 bytes x 180 latitudes x 360 longitudes
> need 1565 Gigabytes.
>
> That's still a bit of memory, so POVEarth is an unreachable dreams for
> still some long years (at least 15, time for the usual computer to
> have 2 tera of Ram).
>
> POVghanistan : south is 29°, west 60°, east 75°, north 39°, that would
> be about 150 tiles , already 3.7 Giga of compact data. But
> transphorming them in mesh is a bad idea.
> Each coordinates/elevation ( 2 bytes in compact format ) is going to
> be a unique 3D coordinates (3 singles, about 12 bytes), and 1 944 000
> 000 ( numbre of points) is nearly half the space of 32 bits index.
> Do not ask why the MeshIndex type is a signed int, but this reduces to
> 31 bits the maximal number of vertices and triangles.
>
> And now come the troubles: each triangle must reference a normal and 3
> vertices. There is approximately 2 triangles per vertex, so there is 3
> 888 000 000 triangles, far too much unless the code is changed to use
> a signed 64 bits as MeshIndex.
> Yet that's only half of the problem: the memory footprint of each
> triangle is enormous (it has to handle smooth triangle, uv mapping and
> texture interpolation, options that might be irrelevant for
> POVghanistan, yet they consumes many bytes per triangle): 69 bytes (if
> I counted right) with 32 bits indexes, 121 bytes with 64 bits indexes.
>
> That's a lot of memory... yet the vector of the mesh are only single
> precision (so down to 7th number), which might be not enough for
> something like POVghanistan. Upgrading to double precision also double
> the memory for vertices' storage.
>
> It might be simpler (sort of) to have an object that behave like a
> sphere with a sea-level-radius and a factor for elevation provided in
> signed-16-bit tiff images at integer longitudes and latitudes: each
> tiff being a sheared (transformation matrix) height-field (rotated to
> match the longitude & latitude on the sphere).
> I'm not sure the compensation of the roundness of earth is necessary
> on each 1°x1° tile, but it could be done later as a post-process when
> generating each height-fields if really needed. (it cannot be done
> with a transform)
>
> Something like:
>
> foobar{ 6371, 0.001
>    tile { 30, 70, "ASTGTM2_N30E070_dem.tif" }
>    tile { 31, 70, "ASTGTM2_N31E070_dem.tif" }
>    tile { 32, 70, "ASTGTM2_N32E070_dem.tif" }
> }
>


-- 

Regards
     Stephen

Post a reply to this message

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Le 17/06/2015 10:59, Stephen a écrit :
> That is a no, then?  ;-)

Up to you.

- From my point of view, a mesh, or many meshes, is not a viable
solution to the original problem.

I suggested one alternative, at least for POVghanistan, that might be
worth considering and coding. It's not obvious but does not seems
impossible.

You might provide other solutions too.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iJwEAQEIAAYFAlWBOK0ACgkQhKAm8mTpkW2+ywQApTqq93IEi7tT6p/tdT4U7XeA
3JBi/uMA4dDV1JOXlBiucxeTiCNJa0Jf5ekBfup5j+SRkYfukYaOodziVs8ii4RC
T9VgQgqb2JdP/gsNl1bhkfhpj69DXtSYOJnMAbscLJzcMvbk7X+g/kXPccNA8noD
zY8O5E+F+Fnz8sr7dh8=
=myGD
-----END PGP SIGNATURE-----

Post a reply to this message

On 17/06/2015 10:06, Le_Forgeron wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Le 17/06/2015 10:59, Stephen a écrit :
>> That is a no, then?  ;-)
>
> Up to you.
>

It would be a no from me.

I've spent many years waiting for computers to catch up with my 
imagination. And when they do. My imagination has moved on to bigger 
things. :-)

> - From my point of view, a mesh, or many meshes, is not a viable
> solution to the original problem.
>

I followed your reasoning. Can't and wouldn't want to fault it.

> I suggested one alternative, at least for POVghanistan, that might be
> worth considering and coding. It's not obvious but does not seems
> impossible.
>
> You might provide other solutions too.
>

I can't. I don't think that it is possible without using adaptive Level 
of Detail. Even if I could code I would not want to try it in SDL.
Although, I suppose you could write an external program and call it from 
PovRay.

Having said that. I am a great admirer of Yadgar's work. (High! Yadgar)
I have been following it forever. In the early days I copied his 
examples, to learn.

I can't help feeling that for modelling such large structures. Digital 
computers have their drawbacks.
But I don't want to go down the philosophy, road. :-)


-- 

Regards
     Stephen

Post a reply to this message

Le 15-06-17 06:31, Stephen a écrit :
> On 17/06/2015 10:06, Le_Forgeron wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Le 17/06/2015 10:59, Stephen a écrit :
>>> That is a no, then?  ;-)
>>
>> Up to you.
>>
>
> It would be a no from me.
>
> I've spent many years waiting for computers to catch up with my
> imagination. And when they do. My imagination has moved on to bigger
> things. :-)
>
>> - From my point of view, a mesh, or many meshes, is not a viable
>> solution to the original problem.
>>
>
> I followed your reasoning. Can't and wouldn't want to fault it.
>
>> I suggested one alternative, at least for POVghanistan, that might be
>> worth considering and coding. It's not obvious but does not seems
>> impossible.
>>
>> You might provide other solutions too.
>>
>
> I can't. I don't think that it is possible without using adaptive Level
> of Detail. Even if I could code I would not want to try it in SDL.
> Although, I suppose you could write an external program and call it from
> PovRay.
>
> Having said that. I am a great admirer of Yadgar's work. (High! Yadgar)
> I have been following it forever. In the early days I copied his
> examples, to learn.
>
> I can't help feeling that for modelling such large structures. Digital
> computers have their drawbacks.
> But I don't want to go down the philosophy, road. :-)
>
>
Why not use a hight_field? A TIFF is one of the supported image formats 
that you can use.

OK, it will be mapped to a plane, but the gray value will be used 
directly as the elevation of each points without the need to convert to 
some other, less compact, format.

Of use a simple mesh, not mesh2. Internaly, they are identical, but the 
mesh may use somewhat less space for it's definition.


Alain

Post a reply to this message

Am 17.06.2015 um 23:46 schrieb Alain:

> Of use a simple mesh, not mesh2. Internaly, they are identical, but the
> mesh may use somewhat less space for it's definition.

Don't. mesh2 parses faster and with less parse-time memory overhead. 
There's no advantage of mesh except that it is easier to hand-code. 
That, and it filters out any duplicate vertices.

Post a reply to this message

Le 15-06-22 12:27, clipka a écrit :
> Am 17.06.2015 um 23:46 schrieb Alain:
>
>> Of use a simple mesh, not mesh2. Internaly, they are identical, but the
>> mesh may use somewhat less space for it's definition.
>
> Don't. mesh2 parses faster and with less parse-time memory overhead.
> There's no advantage of mesh except that it is easier to hand-code.
> That, and it filters out any duplicate vertices.

I stand corrected.

Post a reply to this message