On Facebook you can create 3D posts using an image and a depth map.

Described here:

https://www.facebook.com/help/414295416095269

A couple of questions.

Has anyone created a script to generate these?

Does depth increase linearly? How deep are white and black supposed to 
be? Do these follow geometric rules?

Thanks.

Post a reply to this message

On 6/17/2021 11:30 PM, Mike Horvath wrote:
> Do these follow geometric rules?

I mean, does depth follow inverse square law or something?


Mike

Post a reply to this message

On 6/17/2021 11:30 PM, Mike Horvath wrote:
> On Facebook you can create 3D posts using an image and a depth map.
> 
> Described here:
> 
> https://www.facebook.com/help/414295416095269
> 
> A couple of questions.
> 
> Has anyone created a script to generate these?
> 
> Does depth increase linearly? How deep are white and black supposed to 
> be? Do these follow geometric rules?
> 
> Thanks.


Thinking about it, blackness should be the inverse tangent of the 
distance of an object from the camera. But what about the scale? How 
much is 1 unit? I'm guessing the scale should be somehow related to the 
distance between the viewer's eyes. It's also possible that the Facebook 
plugin fudges all these numbers somehow in completely arbitrary ways.


Mike

Post a reply to this message

Mike Horvath <mik### [at] gmailcom> wrote:
> On 6/17/2021 11:30 PM, Mike Horvath wrote:
> > Do these follow geometric rules?
>
> I mean, does depth follow inverse square law or something?
>
>
> Mike

Hi Mike,

I recommend you to have a look at this article:

http://paulbourke.net/reconstruction/depthmap2/

Hope it will help.

Pascal

Post a reply to this message

On 6/18/2021 12:16 AM, BayashiPascal wrote:
> Mike Horvath <mik### [at] gmailcom> wrote:
>> On 6/17/2021 11:30 PM, Mike Horvath wrote:
>>> Do these follow geometric rules?
>>
>> I mean, does depth follow inverse square law or something?
>>
>>
>> Mike
> 
> Hi Mike,
> 
> I recommend you to have a look at this article:
> 
> http://paulbourke.net/reconstruction/depthmap2/
> 
> Hope it will help.
> 
> Pascal
> 
> 
> 

Interesting. The pigment in that example is clipped and normalized, 
however. Wouldn't it make more sense geometrically to extend the pigment 
to infinity?


Mike

Post a reply to this message

Mike Horvath <mik### [at] gmailcom> wrote:
> On 6/18/2021 12:16 AM, BayashiPascal wrote:
> > Mike Horvath <mik### [at] gmailcom> wrote:
> >> On 6/17/2021 11:30 PM, Mike Horvath wrote:
> >>> Do these follow geometric rules?
> >>
> >> I mean, does depth follow inverse square law or something?
> >>
> >>
> >> Mike
> >
> > Hi Mike,
> >
> > I recommend you to have a look at this article:
> >
> > http://paulbourke.net/reconstruction/depthmap2/
> >
> > Hope it will help.
> >
> > Pascal
> >
> >
> >
>
> Interesting. The pigment in that example is clipped and normalized,
> however. Wouldn't it make more sense geometrically to extend the pigment
> to infinity?
>
>
> Mike

Do you mean the *gradient* is clipped and normalized ? If so, it is done in
order to use it as an entry of the color_map (which afaik only takes value in
[0, 1]). If not, I don't understand your comment.

Pascal

Post a reply to this message

On 6/18/2021 1:19 AM, BayashiPascal wrote:
>> Interesting. The pigment in that example is clipped and normalized,
>> however. Wouldn't it make more sense geometrically to extend the pigment
>> to infinity?
>>
>>
>> Mike
> 
> Do you mean the *gradient* is clipped and normalized ? If so, it is done in
> order to use it as an entry of the color_map (which afaik only takes value in
> [0, 1]). If not, I don't understand your comment.
> 
> Pascal
> 
> 

I mean the gradient function has a minimum and maximum range versus. It 
should more realistically start at the camera and extend into infinity. 
(But also be scaled somehow to the scale of the scene.)

Here is my attempt, though I don't think it is working.

///////////////////////////////////////////////////////////////////////

#declare CAMERAPOS    = <3,3,3>;
#declare CAMERALOOKAT = <0,0,0>;
#declare CAMERAFRONT  = vnormalize(CAMERALOOKAT - CAMERAPOS);
#declare CAMERAFRONTX = CAMERAFRONT.x;
#declare CAMERAFRONTY = CAMERAFRONT.y;
#declare CAMERAFRONTZ = CAMERAFRONT.z;

#declare my_gradient = function(x, y, z, gradx, grady, gradz)
{
	atan(x * gradx + y * grady + z * gradz)/(pi/2)
}

#declare Muns_depth_pigment = pigment
{
	function
	{
//		clipped_scaled_gradient(x, y, z, CAMERAFRONTX, CAMERAFRONTY, 
CAMERAFRONTZ, DEPTHMIN, DEPTHMAX)
		my_gradient(x, y, z, CAMERAFRONTX, CAMERAFRONTY, CAMERAFRONTZ)
	}
	color_map
	{
		[0 color rgb <1,1,1>]
		[1 color rgb <0,0,0>]
	}
	translate CAMERAPOS
}

///////////////////////////////////////////////////////////////////////

Mike

Post a reply to this message

Mike Horvath <mik### [at] gmailcom> wrote:
> On 6/18/2021 1:19 AM, BayashiPascal wrote:
> >> Interesting. The pigment in that example is clipped and normalized,
> >> however. Wouldn't it make more sense geometrically to extend the pigment
> >> to infinity?
> >>
> >>
> >> Mike
> >
> > Do you mean the *gradient* is clipped and normalized ? If so, it is done in
> > order to use it as an entry of the color_map (which afaik only takes value in
> > [0, 1]). If not, I don't understand your comment.
> >
> > Pascal
> >
> >
>
> I mean the gradient function has a minimum and maximum range versus. It
> should more realistically start at the camera and extend into infinity.
> (But also be scaled somehow to the scale of the scene.)
>
> Here is my attempt, though I don't think it is working.
>
> ///////////////////////////////////////////////////////////////////////
>
> #declare CAMERAPOS    = <3,3,3>;
> #declare CAMERALOOKAT = <0,0,0>;
> #declare CAMERAFRONT  = vnormalize(CAMERALOOKAT - CAMERAPOS);
> #declare CAMERAFRONTX = CAMERAFRONT.x;
> #declare CAMERAFRONTY = CAMERAFRONT.y;
> #declare CAMERAFRONTZ = CAMERAFRONT.z;
>
> #declare my_gradient = function(x, y, z, gradx, grady, gradz)
> {
>  atan(x * gradx + y * grady + z * gradz)/(pi/2)
> }
>
> #declare Muns_depth_pigment = pigment
> {
>  function
>  {
> //  clipped_scaled_gradient(x, y, z, CAMERAFRONTX, CAMERAFRONTY,
> CAMERAFRONTZ, DEPTHMIN, DEPTHMAX)
>   my_gradient(x, y, z, CAMERAFRONTX, CAMERAFRONTY, CAMERAFRONTZ)
>  }
>  color_map
>  {
>   [0 color rgb <1,1,1>]
>   [1 color rgb <0,0,0>]
>  }
>  translate CAMERAPOS
> }
>
> ///////////////////////////////////////////////////////////////////////
>
> Mike

I don't see any problem with your solution. It really just depends on how you
(or the tool you use) interpret the generated depth values.

I've never made such images for Facebook, and have no idea how their plugin
works. Googling a little about it brought me to this article:
https://techcrunch.com/2018/06/07/how-facebooks-new-3d-photos-work/
containing a link to the research paper behind the plugin:
http://visual.cs.ucl.ac.uk/pubs/instant3d/
I had a quick look at it but had neither the courage nor motivation to read it
entirely. However I've noticed page 5 they said "However, this
did not achieve good results, because, as we learned, many depth
maps are normalized using unknown curves. We tried a variety of
other classes of global transformations, ...". Which makes me think they're
reconstructing an approximate 3D mesh, pleasant for the eye but not realistic,
to manage any kind of input, noise, and extrapolation from even a single image.
After all, their goal was probably to make an user-friendly plugin, rather than
accurate.

Conclusion, it probably doesn't really matter what kind of depth encoding you
use if you plan to use it on Facebook ! Maybe the best is to experiment...

I've wrote the script on the site of Paul Bourke while working with 3D scanners
where depth was encoding linearly either planar or spherical distance from the
camera within a range. You can also set DEPTHMIN to 0.0 and DEPTHMAX to an
arbitrarily big value to get a depth map from the camera to (kind of) infinity.
That's why they are variables and not hard coded in my script.

Pascal

Post a reply to this message

On 6/18/2021 2:51 AM, BayashiPascal wrote:
> Conclusion, it probably doesn't really matter what kind of depth encoding you
> use if you plan to use it on Facebook ! Maybe the best is to experiment...

Interesting.

I wonder if there is a tool that works with stereo images that is less 
ambiguous?

But it might not matter.


Mike

Post a reply to this message

Mike Horvath <mik### [at] gmailcom> wrote:
> On 6/18/2021 2:51 AM, BayashiPascal wrote:
> > Conclusion, it probably doesn't really matter what kind of depth encoding you
> > use if you plan to use it on Facebook ! Maybe the best is to experiment...
>
> Interesting.
>
> I wonder if there is a tool that works with stereo images that is less
> ambiguous?
>
> But it might not matter.
>
>
> Mike

This field is called 'photogrammetry' ... and it's a rabbit hole, I warn you (I
work on it every day) !!

Maybe you can have look at the wikipedia page first if you don't know about it.
https://en.wikipedia.org/wiki/Photogrammetry

There are tons of techniques and tools, each with its particular advantage,
inconvenient and use case...

Pascal

Post a reply to this message

Here is a tutorial on how to generate a depth map from stereo images:

https://www.docs.opencv.org/master/dd/d53/tutorial_py_depthmap.html

However, the accuracy does not look very good. I suspect there is also a 
bias in accuracy in one (the horizontal?) direction.

I may have to do this for a few of my scenes, though, as overriding all 
of my textures just to create a depth map is a serious PITA. (There's no 
easy way to do it since povray SDL is not OOP.)


Mike

Post a reply to this message

Two examples:

https://www.facebook.com/groups/fansofpovray/permalink/4181681918565652/

https://www.facebook.com/groups/fansofpovray/permalink/4185621304838380/

You need to have an FB account and be a member of the group in order to 
see them (I think).

One turned out okay; the other was a complete failure.


Mike

Post a reply to this message

On 6/19/2021 3:43 PM, Mike Horvath wrote:
> the other was a complete failure.
> 

Well, not a total failure, but it came out pretty bad.


Mike

Post a reply to this message

"BayashiPascal" <bai### [at] gmailcom> wrote:

> This field is called 'photogrammetry' ... and it's a rabbit hole, I warn you (I
> work on it every day) !!

Have you implemented any of that work in POV-Ray?  I think even some of the
basics would encourage others to learn more about this intriguing field!

Post a reply to this message

"Bald Eagle" <cre### [at] netscapenet> wrote:
> "BayashiPascal" <bai### [at] gmailcom> wrote:
>
> > This field is called 'photogrammetry' ... and it's a rabbit hole, I warn you (I
> > work on it every day) !!
>
> Have you implemented any of that work in POV-Ray?  I think even some of the
> basics would encourage others to learn more about this intriguing field!

I did, but not in POV-Ray. POV-Ray is a rendering tool, it takes as input a
representation of a 3D scene and output a 2D image. Photogrammetry does the
opposite, it takes 2D image(s) as input and outputs a representation of the
corresponding 3D scene. So, I don't see how one could implements photogrammetry
in POV-Ray, they serve both two opposite purposes.

In this thread, Mike Horvath was initially speaking about how to create depth
image using POV-Ray. The depth image is the input of the photogrammetry
algorithm used in Facebook, which would otherwise be acquired using a 3D scanner
for example. This depth image can be generated as described in the link I gave
earlier. But this is really just about generating an input for a certain type of
photogrammetry algorithm, not about performing photogrammetry itself.

Let me know if I misunderstand your question, I'll be happy to correct my reply.

Pascal

Post a reply to this message

> I did, but not in POV-Ray. POV-Ray is a rendering tool, it takes as input a
> representation of a 3D scene and output a 2D image. Photogrammetry does the
> opposite, it takes 2D image(s) as input and outputs a representation of the
> corresponding 3D scene. So, I don't see how one could implements photogrammetry
> in POV-Ray, they serve both two opposite purposes.

Yes, but we don't always start with a blank slate and start coding a scene in a
vacuum.  We have all sort of "input data" that we can use for POV-Ray's SDL
language to perform operations upon.  And so sometimes we need to take a 2D
image, reverse the process to obtain the relevant 3D information, and then use
that to render a new 2D image.

(1)
Take for example Francois LE COAT's very interesting work:
http://news.povray.org/povray.advanced-users/thread/%3Cweb.5bb77cec1f36de80c437ac910%40news.povray.org%3E/

Forgive me if there's a lot of things I don't understand, properly recall, or
have wrong, but it seems to me that an aerial photograph of a landscape has a
lot of perspective distortion, and part of the photogrammetry process would be
correcting for that.

(2)
If I wanted to take a photograph of a piece of writing paper laying on a table
and use the portion of the image with the paper in it as an image_map, I'd
likely have a horribly non-rectangular area to cope with.  Preumably there are
photogrammetry-related tricks to deduce what sort of transformation matrix I
would need to apply in order to mimic reorienting the paper to be perpendicular
to the camera, and have straight edges with 90-degree corners.

(3)
Let me preface this one with some information that will help you understand what
I'm talking about:

http://imcs.dvfu.ru/lib.int/docs/Programming/Graphics/Academic Press Graphics
Gems Ii 1995.pdf
(use the wayback machine to get the PDF.   pg 181 (pg 208 of pdf) )

View correlation.
"To combine computer-generated objects into a photographic scene, it is
necessary to render the objects from the same point of view as was used
to make the photo."


Rendering Synthetic Objects into Legacy Photographs
Kevin Karsch Varsha Hedau David Forsyth Derek Hoiem
University of Illinois at Urbana-Champaign
{karsch1,vhedau2,daf,dhoiem}@uiuc.edu
https://web.archive.org/web/20190211102945/http://kevinkarsch.com/publications/sa11-lowres.pdf

or:
https://www.popsci.com/technology/article/2011-10/new-program-slips-super-accurate-false-images-existing-photographs

https://www.kevinkarsch.com/?page_id=445

https://arxiv.org/abs/2001.00986


There are plenty of examples of people creating POV-Ray renderings based on real
things - things they often only have photographs of.

But then I was thinking that there are probably actual POV-Ray renderings, but
somehow the source to generate those renderings have been lost due to the code
never being posted, a HDD crash, or just getting - lost.  People might want to
recreate a scene, and having some basic information about the size and placement
of the objects in the scene would greatly speed up the writing of a new scene
file.

I know that I have done some work to recreate some of the documentation images
for things like isosurfaces, and didn't have the code for those images.   I had
to make educated guesses.  I "knew" the probably size of the object, or at least
its relative scale, and then I just needed to place the camera and light source
in the right place to get the image to look the same.   But determining where
the camera and light source are seems to me to be something that could be
calculated using "photogrammetric image cues" and well-established equations.

Let's take a photograph of a room.  It likely has tables and chairs and windows,
and these all have right angles and typical sizes.  It seems to me that there
might be a way to use photogrammetry to compute the 3D points of the corners and
rapidly generate a basic set of vectors for the proper sizing and placement of
everything in a basic rendered version of that same room.

I know that they have cell phone apps that can generate a floor plan of your
house just by snapping a few photos of the rooms from different angles.

Also, the "augmented reality" apps for cell phones.



I think there are a lot of cool things that could be done with POV-Ray in the
"opposite direction" of what people normally do, and the results could be very
interesting, very educational, and give rise to tools that would probably find
every day usage for generated new scenes, and perhaps serve to attract new
users.

It was just my thought that if you do this "every day", that you would have the
level of understanding to quickly implement at least some of the basics, whereas
I'd probably spend the next few months chasing my tail and tying equations up
into Gordian knots before I finally understoof what I was doing wrong.

Post a reply to this message

> ...

Thanks for your precision. I'll reply to you within a few days.

Pascal

Post a reply to this message

On 6/27/2021 8:19 AM, Bald Eagle wrote:
> (1)
> Take for example Francois LE COAT's very interesting work:
>
http://news.povray.org/povray.advanced-users/thread/%3Cweb.5bb77cec1f36de80c437ac910%40news.povray.org%3E/
> 
> Forgive me if there's a lot of things I don't understand, properly recall, or
> have wrong, but it seems to me that an aerial photograph of a landscape has a
> lot of perspective distortion, and part of the photogrammetry process would be
> correcting for that.
> 

Unless you use an orthographic camera. But IIRC they only work well for 
extremely short distances.


Mike

Post a reply to this message

On 6/27/2021 8:19 AM, Bald Eagle wrote:
> There are plenty of examples of people creating POV-Ray renderings based on real
> things - things they often only have photographs of.
> 
> But then I was thinking that there are probably actual POV-Ray renderings, but
> somehow the source to generate those renderings have been lost due to the code
> never being posted, a HDD crash, or just getting - lost.  People might want to
> recreate a scene, and having some basic information about the size and placement
> of the objects in the scene would greatly speed up the writing of a new scene
> file.
> 
> I know that I have done some work to recreate some of the documentation images
> for things like isosurfaces, and didn't have the code for those images.   I had
> to make educated guesses.  I "knew" the probably size of the object, or at least
> its relative scale, and then I just needed to place the camera and light source
> in the right place to get the image to look the same.   But determining where
> the camera and light source are seems to me to be something that could be
> calculated using "photogrammetric image cues" and well-established equations.
> 
> Let's take a photograph of a room.  It likely has tables and chairs and windows,
> and these all have right angles and typical sizes.  It seems to me that there
> might be a way to use photogrammetry to compute the 3D points of the corners and
> rapidly generate a basic set of vectors for the proper sizing and placement of
> everything in a basic rendered version of that same room.
> 
> I know that they have cell phone apps that can generate a floor plan of your
> house just by snapping a few photos of the rooms from different angles.
> 
> Also, the "augmented reality" apps for cell phones.
> 

Can (most likely) already be done using general purpose external tools. 
At what stage is povray useful/necessary?


Mike

Post a reply to this message

Hi,

Bald Eagle writes:
> Take for example Francois LE COAT's very interesting work:
> 
> http://news.povray.org/povray.advanced-users/thread/%3Cweb.5bb77cec1f36
de80c437ac910%40news.povray.org%3E/
> 
> Forgive me if there's a lot of things I don't understand, properly reca
ll, or
> have wrong, but it seems to me that an aerial photograph of a landscape
 has a
> lot of perspective distortion, and part of the photogrammetry process w
ould be
> correcting for that.

For the time being, I'm experimenting to model trajectories in 3D...

	<https://www.youtube.com/watch?v=uQEo7fD0GAo>

...but it is a difficult work, because public images given by the NASA
are not of good quality. If the Ingenuity helicopter flying on Mars was
driven with my trajectory modelling, it would have crashed. But it is
what I'm measuring, with input data I could collect. This is strange!
I have no other possibility than to improve my computations further on.

Best regards,

-- 
François LE COAT
<http://eureka.atari.org/>

Post a reply to this message

@baldeagle, sorry for the late reply.

> Yes, but we don't always start with a blank slate and start coding a scene in a
> vacuum.  We have all sort of "input data" that we can use for POV-Ray's SDL
> language to perform operations upon.  And so sometimes we need to take a 2D
> image, reverse the process to obtain the relevant 3D information, and then use
> that to render a new 2D image.

Yes, I agree.

> Forgive me if there's a lot of things I don't understand, properly recall, or
> have wrong, but it seems to me that an aerial photograph of a landscape has a
> lot of perspective distortion, and part of the photogrammetry process would be
> correcting for that.
> If I wanted to take a photograph of a piece of writing paper laying on a table
> and use the portion of the image with the paper in it as an image_map, I'd
> likely have a horribly non-rectangular area to cope with.  Preumably there are
> photogrammetry-related tricks to deduce what sort of transformation matrix I
> would need to apply in order to mimic reorienting the paper to be perpendicular
> to the camera, and have straight edges with 90-degree corners.

Yes, you're right.

> Let me preface this one with some information that will help you understand what
> I'm talking about:

Thanks for the links.

> There are plenty of examples of people creating POV-Ray renderings based on real
> things - things they often only have photographs of.

Yes, that's exactly one of the things I'm doing at work. (I've used
photogrammetry to create 3D models of archeological artifacts for few years, and
I'm know working on the development of photogrammetric algorithms applied to
AI).

> But then I was thinking that there are probably actual POV-Ray renderings, but
> somehow the source to generate those renderings have been lost due to the code
> never being posted, a HDD crash, or just getting - lost.  People might want to
> recreate a scene, and having some basic information about the size and placement
> of the objects in the scene would greatly speed up the writing of a new scene
> file.

Sure, that's a legitimate motivation.

> I know that I have done some work to recreate some of the documentation images
> for things like isosurfaces, and didn't have the code for those images.   I had
> to make educated guesses.  I "knew" the probably size of the object, or at least
> its relative scale, and then I just needed to place the camera and light source
> in the right place to get the image to look the same.   But determining where
> the camera and light source are seems to me to be something that could be
> calculated using "photogrammetric image cues" and well-established equations.
> Let's take a photograph of a room.  It likely has tables and chairs and windows,
> and these all have right angles and typical sizes.  It seems to me that there
> might be a way to use photogrammetry to compute the 3D points of the corners and
> rapidly generate a basic set of vectors for the proper sizing and placement of
> everything in a basic rendered version of that same room.

Yes, and no. Let's take the example of the coordinates of the corner of your
table. First, you have to know that's a corner of the table. It can be specify
by the user manually, or done by automatic detection which works more or less
depending on the image. Once you have the 2D coordinates in the image, in order
to get the 3D coordinates you actually need several images (and identify that
particular corner in each of them) and the intrinsic and extrinsic parameters of
the camera. If you have none of these, you can try with algorithms like the one
used on Facebook, and you can see what kind of result to expect with the tests
by Mike. If you have only part of these, there are other solutions, depending on
what you have and giving more or less accurate results. These are all very
complicated and I'm not going to explain them here. Then, even if you have your
3D coordinates of corners, how do you know which corner relates to which other
corner ? You, as a human, can guess it by looking at the picture, but if you
expect something entirely automated, you have to use or implement yet other very
complex technics. Then, let say you've found your way up to an approximate 3D
model of the geometry (not speaking of the fact that you'll get one big mesh for
the whole scene, if you want to split by logical entities or get POV-Ray
primitives you'll have to find a way to identity and convert them), next comes
the texture. Textures are influenced by the lights (ambient and locals), the
camera sensor, each other textures through radiosity, even the eye of each
individual (of which I'm well aware, having deuteranopia)... About finding the
light position, in a real environment I hope you understand now that's barely
feasible in an fully automated way. Even if your input image is a simple
rendered image with a single pointlight ligth_source and no radiosity, would you
expect to guess the position from the shadows ? Again, you as a human, having a
high level of comprehension of the scene you're looking at, it makes sense, but
for an algorithm it really doesn't. Consider this simple question: if you see a
cylinder with a dark side on the left and a bright side on the right, is this
because the light was on the right, or because the cylinder happens to have such
a texture that it looks that way even when light up from the left ?

> I know that they have cell phone apps that can generate a floor plan of your
> house just by snapping a few photos of the rooms from different angles.

Yes, and from my experience these apps are all clumsy and inaccurate at best. I
know there are a lot of mesmerizing videos on Youtube about such apps. To me,
these are just clickbait and commercials. When you have to do it every days
under real conditions and looking for professional level accurate results, it
takes a lot of experience, preprocessing and postprocessing work, which are
rarely spoken of in those videos.

> Also, the "augmented reality" apps for cell phones.

Augmented reality is a bit different in my opinion. While some applications may
create content that seems to be integrated into the 3D scene, they actually all
works at the 2D level, adding contents directly in the displayed image,
eventually using some 3D cues from the underlying image. But I've never worked
with augmented reality technics so I may be wrong.

> I think there are a lot of cool things that could be done with POV-Ray in the
> "opposite direction" of what people normally do, and the results could be very
> interesting, very educational, and give rise to tools that would probably find
> every day usage for generated new scenes, and perhaps serve to attract new
> users.
> It was just my thought that if you do this "every day", that you would have the
> level of understanding to quickly implement at least some of the basics, whereas
> I'd probably spend the next few months chasing my tail and tying equations up
> into Gordian knots before I finally understoof what I was doing wrong.

In a sense that's what I'm doing, having convinced my employer that we could use
a combination of POV-Ray and photogrammetry to help solving a certain AI
problem. But it's already been 1.5 years of R&D and still a WIP, so when it
comes to "quickly implement some of the basics", I hope you now understand that
even the basics are far from simple and definitely not quick to implement. I'm
really sorry but writing tutorials on the subject would be a way too big project
for me to embark on (just replying to your message took me a week!).

If you want to try photogrammetry by yourself, first I would recommend to use
already available softwares, and have modest expectation about the result,
instead of trying to implement anything yourself. I've used a lot Agisoft
Metashape (https://www.agisoft.com/) and recommend it. On the free open-source
side, have a look at Meshroom (https://alicevision.org/#meshroom). If you really
wants to implement something by yourself, Open3D could be a good start point and
they have some tutorials available
(http://www.open3d.org/docs/latest/tutorial/Basic/index.html).

Hoping that will help!

Pascal

Post a reply to this message

Thanks for that - very detailed - explanation. It reminds me of two 
things (or rather three):

1) You may remember my POV-Ray scene 'Paris la nuit' a couple of years 
ago, based on a photograph by Sabine Weiss in 1953. It was just done by 
trial and error of course, and a hell of a challenge with all kind of 
assumptions. Nothing to do with photogrammetry of course, but I 
appreciate your caveats about any "easy magic" ;-).

2) I have seen on a couple of occasions (on TV), archaeologists make a 
lot of photographs of an object, under all kind of angles, and later 
combine those into a 3d model (with software of course). I saw that done 
  in particular on the terracotta army in China. Closer to home, geology 
students recently used a drone to photograph the walls of a quarry in 
the same manner, and assembled them into a 3d model of the quarry. 
Fascinating stuff, and relatively cheap to implement, especially for 
students I understood.

3) I have been interested in archaeology for most of my life and so came 
quite early across the use of photogrammetry there. If I remember well, 
it was used by Unesco during the construction of the Assouan Dam in 
Egypt to move the Abou Simbel temple to a higher position. I was a 
reader of 'Archeologia' at the time.

-- 
Thomas

Post a reply to this message

On 7/4/2021 2:20 AM, Thomas de Groot wrote:
> 2) I have seen on a couple of occasions (on TV), archaeologists make a 
> lot of photographs of an object, under all kind of angles, and later 
> combine those into a 3d model (with software of course). I saw that done 
>   in particular on the terracotta army in China. Closer to home, geology 
> students recently used a drone to photograph the walls of a quarry in 
> the same manner, and assembled them into a 3d model of the quarry. 
> Fascinating stuff, and relatively cheap to implement, especially for 
> students I understood.
> 

I wonder *how* cheap it really is. Probably *not* cheap, in terms of the 
work and expertise involved.


Mike

Post a reply to this message

Op 4-7-2021 om 11:09 schreef Mike Horvath:
> On 7/4/2021 2:20 AM, Thomas de Groot wrote:
>> 2) I have seen on a couple of occasions (on TV), archaeologists make a 
>> lot of photographs of an object, under all kind of angles, and later 
>> combine those into a 3d model (with software of course). I saw that 
>> done   in particular on the terracotta army in China. Closer to home, 
>> geology students recently used a drone to photograph the walls of a 
>> quarry in the same manner, and assembled them into a 3d model of the 
>> quarry. Fascinating stuff, and relatively cheap to implement, 
>> especially for students I understood.
>>
> 
> I wonder *how* cheap it really is. Probably *not* cheap, in terms of the 
> work and expertise involved.
> 
I don't really know. What I understood, in particular from the Chinese 
example, was that the hardware came 'cheap' as only a perfectly common 
digital camera was needed and no sophisticated laser-controlled stuff. 
And a simple stepladder in addition, to get around the statues. The same 
applied for the students with their drone. Concerning the software, I 
have no idea. The results looked good however, and I doubt that they had 
any particularly high expertise in the matter.

-- 
Thomas

Post a reply to this message

> 1) You may remember my POV-Ray scene 'Paris la nuit' a couple of years
> ago, based on a photograph by Sabine Weiss in 1953. It was just done by
> trial and error of course, and a hell of a challenge with all kind of
> assumptions. Nothing to do with photogrammetry of course, but I
> appreciate your caveats about any "easy magic" ;-).

I can't recall it from the name, neither find it with Google. Would you have a
link ? I would certainly enjoy seeing it again, as usual with your scenes :-)

> 2) I have seen on a couple of occasions (on TV), archaeologists make a
> lot of photographs of an object, under all kind of angles, and later
> combine those into a 3d model (with software of course). I saw that done
>   in particular on the terracotta army in China. Closer to home, geology
> students recently used a drone to photograph the walls of a quarry in
> the same manner, and assembled them into a 3d model of the quarry.
> Fascinating stuff, and relatively cheap to implement, especially for
> students I understood.

Yes, it's heavily used in archaeology, generally only for the most important
pieces as it is very time consuming, and sometime challenging. You will probably
enjoy this speech of an archaeologist talking about their struggles to produce
models of obsidian artifacts:
https://www.youtube.com/watch?v=g0YaWDrl5qI
I haven't mention it in my previous post but, any reflective or transparent
texture is also an immediate 'no go'. It completely confuses all current
algorithms.

> 3) I have been interested in archaeology for most of my life and so came
> quite early across the use of photogrammetry there. If I remember well,
> it was used by Unesco during the construction of the Assouan Dam in
> Egypt to move the Abou Simbel temple to a higher position. I was a
> reader of 'Archeologia' at the time.

I had plan to become a paleobiologist until I entered university where I've been
reoriented toward computer science. This has probably been a wise advice but I
always wonder what would have become that other me. So, I've been really happy
when I had the chance to work for archaeologists a few years ago.

Post a reply to this message

Thomas de Groot <tho### [at] degrootorg> wrote:
> Op 4-7-2021 om 11:09 schreef Mike Horvath:
> > On 7/4/2021 2:20 AM, Thomas de Groot wrote:
> >> 2) I have seen on a couple of occasions (on TV), archaeologists make a
> >> lot of photographs of an object, under all kind of angles, and later
> >> combine those into a 3d model (with software of course). I saw that
>
> >> geology students recently used a drone to photograph the walls of a
> >> quarry in the same manner, and assembled them into a 3d model of the
> >> quarry. Fascinating stuff, and relatively cheap to implement,
> >> especially for students I understood.
> >>
> >
> > I wonder *how* cheap it really is. Probably *not* cheap, in terms of the
> > work and expertise involved.
> >
> I don't really know. What I understood, in particular from the Chinese
> example, was that the hardware came 'cheap' as only a perfectly common
> digital camera was needed and no sophisticated laser-controlled stuff.
> And a simple stepladder in addition, to get around the statues. The same
> applied for the students with their drone. Concerning the software, I
> have no idea. The results looked good however, and I doubt that they had
> any particularly high expertise in the matter.
>
> --
> Thomas


You can indeed produce good results with a standard camera. The software itself,
if not freeware is quite expensive, but if they were students it's probably
provided by their university and that's surely not much compared to a university
budget. About expertise, if they were students, that very probably mean they had
teachers with a high level of expertise to guide them.

Post a reply to this message

"BayashiPascal" <bai### [at] gmailcom> wrote:
> @baldeagle, sorry for the late reply.

No apologies necessary, as it's a complicated topic, and I'm sure you have
PLENTY to keep you busy IRL.

> Yes, and no. Let's take the example of the coordinates of the corner of your
> table. First, you have to know that's a corner of the table. It can be specify
> by the user manually, or done by automatic detection which works more or less
> depending on the image. Once you have the 2D coordinates in the image, in order
> to get the 3D coordinates you actually need several images (and identify that
> particular corner in each of them) and the intrinsic and extrinsic parameters of
> the camera.

This is certainly what I would have expected from my )all too brief) readings of
the photogrammetry literature, which is why I was so surprised when I came
across the work of Kevin Karsch.  It seems like "magic" - certainly very bold
claims, and I was wondering if
a) you could look deeper into his work, with your education and experience in
the area and see how much of what he claims to accomplish is "embellished"
and
b) if he and his colleagues/coworkers/students have done a lot of the puzzling
out and heavy lifting already, then perhaps he might provide you with tools that
would help you in your own work that you might not be familiar with.

I certainly didn't want to send you "off into the weeds" without thinking that
there might be some personal/professional benefit for you in the process.

> Then, even if you have your
> 3D coordinates of corners, how do you know which corner relates to which other
> corner ?

Dr Karsch apparently has most of that figured out, and even has the code (mostly
..js) posted on his site.  He welcomes someone to get it up and running on a
server again, and if we have a web guru amongst us, I can envision using his
work as a basis for better understanding what he did, converting some of that
javascript to SDL / source code, and perhaps even spurring some further
development of a POV-Ray modeler.

I am not worried about the texture, as that is separate from the geometry.

> About finding the
> light position, in a real environment I hope you understand now that's barely
> feasible in an fully automated way.

Again, this is part of the series of bold claims that Karsch makes and
apparently demonstrates in his papers, software, and videos.  (He does mention
that a database is used, and references an open source DB)


> Yes, and from my experience these apps are all clumsy and inaccurate at best. I
> know there are a lot of mesmerizing videos on Youtube about such apps. To me,
> these are just clickbait and commercials. When you have to do it every days
> under real conditions and looking for professional level accurate results, it
> takes a lot of experience, preprocessing and postprocessing work, which are
> rarely spoken of in those videos.

Right - "the more you know"...  And I'm fine with a lot of what I've come across
in the past few years being declared to be fantastic claims that mislead the
reader/viewer.

> In a sense that's what I'm doing, having convinced my employer that we could use
> a combination of POV-Ray and photogrammetry to help solving a certain AI
> problem. But it's already been 1.5 years of R&D and still a WIP, so when it
> comes to "quickly implement some of the basics", I hope you now understand that
> even the basics are far from simple and definitely not quick to implement.

Certainly.  I understand that, and was in no way expecting a tutorial of any
kind.

> I'm
> really sorry but writing tutorials on the subject would be a way too big project
> for me to embark on (just replying to your message took me a week!).


Besides the explicitly photogrammetric aspect of the above where the first step
is acquiring key vertices of the geometry in the image, I was thinking it would
be much simpler, straightforward, and immediately useful if any POV-Ray work
were limited to what to do with vertex data already in possession of the user.

What if I took the uv coordinates or the interpolated <x,y,z> coordinates from
the screen position-finder macro

http://news.povray.org/povray.binaries.images/thread/%3C5baddad1%40news.povray.org%3E/?mtop=424795
or the modifications to the screen.inc macros
http://news.povray.org/povray.binaries.scene-files/thread/%3C4afccd8a%241%40news.povray.org%3E/

and worked from there, without having to worry about all the messy data
extraction from a photograph?   I think the basic geometric transformations
would be instructive and useful all on their own.

I mean, you're already familiar with the issues regarding the FieldCam macro.
http://news.povray.org/povray.binaries.images/thread/%3C5fb22ee0%40news.povray.org%3E/?mtop=432106

I mean, I was able to measure and guess when I was recreating this challenging
piece:
https://secureservercdn.net/45.40.146.28/ik7.9d0.myftpupload.com/wp-content/uploads/revslider/home-part/hero_part_with_
circles.png

But perhaps there is a way to embed specially colored pixels into an image at
key reference points, like they do with steganography?  (thinking for testing,
and future recovery of scene geometry if it is lost) Then a scan for vertices
could be done with the raw pixel data, and it would be even easier.   Also, a
user could mark a copy of the photo with an image editor before a macro
processed it.

Just some ideas for what we could do with existing povray technology and some
educated application of matrix transforms.


Thanks for your caveats and opinions.   Hopefully some of what I posted can help
you in your own work.  I am always interested in hearing about what challenges
you experience in trying to solve your own geometric problems - it's an
interesting field.

- Bill

Post a reply to this message

Op 4-7-2021 om 14:36 schreef BayashiPascal:
>> 1) You may remember my POV-Ray scene 'Paris la nuit' a couple of years
>> ago, based on a photograph by Sabine Weiss in 1953. It was just done by
>> trial and error of course, and a hell of a challenge with all kind of
>> assumptions. Nothing to do with photogrammetry of course, but I
>> appreciate your caveats about any "easy magic" ;-).
> 
> I can't recall it from the name, neither find it with Google. Would you have a
> link ? I would certainly enjoy seeing it again, as usual with your scenes :-)
> 
It was made for one of the TC-RTC Challenges. Since Stephen and I closed 
that down, it has also disappeared from the web.

I shall repost the image with some comments I wrote at the time; I hope 
to find that back in my archives...

>> 2) I have seen on a couple of occasions (on TV), archaeologists make a
>> lot of photographs of an object, under all kind of angles, and later
>> combine those into a 3d model (with software of course). I saw that done
>>    in particular on the terracotta army in China. Closer to home, geology
>> students recently used a drone to photograph the walls of a quarry in
>> the same manner, and assembled them into a 3d model of the quarry.
>> Fascinating stuff, and relatively cheap to implement, especially for
>> students I understood.
> 
> Yes, it's heavily used in archaeology, generally only for the most important
> pieces as it is very time consuming, and sometime challenging. You will probably
> enjoy this speech of an archaeologist talking about their struggles to produce
> models of obsidian artifacts:
> https://www.youtube.com/watch?v=g0YaWDrl5qI
> I haven't mention it in my previous post but, any reflective or transparent
> texture is also an immediate 'no go'. It completely confuses all current
> algorithms.
> 
Yes, I can understand that !

>> 3) I have been interested in archaeology for most of my life and so came
>> quite early across the use of photogrammetry there. If I remember well,
>> it was used by Unesco during the construction of the Assouan Dam in
>> Egypt to move the Abou Simbel temple to a higher position. I was a
>> reader of 'Archeologia' at the time.
> 
> I had plan to become a paleobiologist until I entered university where I've been
> reoriented toward computer science. This has probably been a wise advice but I
> always wonder what would have become that other me. So, I've been really happy
> when I had the chance to work for archaeologists a few years ago.
> 
Who knows? I wonder sometimes about my life in an alternate universe ;-] 
  But life has sometimes a high degree of serendipity.

-- 
Thomas

Post a reply to this message

Op 4-7-2021 om 14:36 schreef BayashiPascal:
>> 1) You may remember my POV-Ray scene 'Paris la nuit' a couple of years
>> ago, based on a photograph by Sabine Weiss in 1953. It was just done by
>> trial and error of course, and a hell of a challenge with all kind of
>> assumptions. Nothing to do with photogrammetry of course, but I
>> appreciate your caveats about any "easy magic" ;-).
> 
> I can't recall it from the name, neither find it with Google. Would you have a
> link ? I would certainly enjoy seeing it again, as usual with your scenes :-)
> 
There is this:

http://news.povray.org/povray.binaries.images/thread/%3C4d8dea6b%40news.povray.org%3E/?mtop=359553

-- 
Thomas

Post a reply to this message

Op 4-7-2021 om 17:32 schreef Thomas de Groot:
> Op 4-7-2021 om 14:36 schreef BayashiPascal:
>>> 1) You may remember my POV-Ray scene 'Paris la nuit' a couple of years
>>> ago, based on a photograph by Sabine Weiss in 1953. It was just done by
>>> trial and error of course, and a hell of a challenge with all kind of
>>> assumptions. Nothing to do with photogrammetry of course, but I
>>> appreciate your caveats about any "easy magic" ;-).
>>
>> I can't recall it from the name, neither find it with Google. Would 
>> you have a
>> link ? I would certainly enjoy seeing it again, as usual with your 
>> scenes :-)
>>
> There is this:
> 
>
http://news.povray.org/povray.binaries.images/thread/%3C4d8dea6b%40news.povray.org%3E/?mtop=359553

> 
> 
There is also this. A later version:

http://news.povray.org/povray.binaries.images/thread/%3C5b7a6cb0%40news.povray.org%3E/?ttop=428926&toff=150&mtop=424347

-- 
Thomas

Post a reply to this message

Thomas de Groot <tho### [at] degrootorg> wrote:
> Op 4-7-2021 om 17:32 schreef Thomas de Groot:
> > Op 4-7-2021 om 14:36 schreef BayashiPascal:
> >>> 1) You may remember my POV-Ray scene 'Paris la nuit' a couple of years
> >>> ago, based on a photograph by Sabine Weiss in 1953. It was just done by
> >>> trial and error of course, and a hell of a challenge with all kind of
> >>> assumptions. Nothing to do with photogrammetry of course, but I
> >>> appreciate your caveats about any "easy magic" ;-).
> >>
> >> I can't recall it from the name, neither find it with Google. Would
> >> you have a
> >> link ? I would certainly enjoy seeing it again, as usual with your
> >> scenes :-)
> >>
> > There is this:
> >
> >
http://news.povray.org/povray.binaries.images/thread/%3C4d8dea6b%40news.povray.org%3E/?mtop=359553
> >
> >
> There is also this. A later version:
>
>
http://news.povray.org/povray.binaries.images/thread/%3C5b7a6cb0%40news.povray.org%3E/?ttop=428926&toff=150&mtop=4243
47
>
> --
> Thomas

Thanks. A very faithful reproduction of the original, very nice work :-)

Post a reply to this message

"Bald Eagle" <cre### [at] netscapenet> wrote:
....

To be honest I haven't read the link you've provided about the works of Pr
Kasch, by lack of time. But I'll try to do so and come back to you this week.

Pascal

Post a reply to this message

Op 5-7-2021 om 05:30 schreef BayashiPascal:
> Thomas de Groot <tho### [at] degrootorg> wrote:
>> Op 4-7-2021 om 17:32 schreef Thomas de Groot:
>>> Op 4-7-2021 om 14:36 schreef BayashiPascal:
>>>>> 1) You may remember my POV-Ray scene 'Paris la nuit' a couple of years
>>>>> ago, based on a photograph by Sabine Weiss in 1953. It was just done by
>>>>> trial and error of course, and a hell of a challenge with all kind of
>>>>> assumptions. Nothing to do with photogrammetry of course, but I
>>>>> appreciate your caveats about any "easy magic" ;-).
>>>>
>>>> I can't recall it from the name, neither find it with Google. Would
>>>> you have a
>>>> link ? I would certainly enjoy seeing it again, as usual with your
>>>> scenes :-)
>>>>
>>> There is this:
>>>
>>>
http://news.povray.org/povray.binaries.images/thread/%3C4d8dea6b%40news.povray.org%3E/?mtop=359553
>>>
>>>
>> There is also this. A later version:
>>
>>
http://news.povray.org/povray.binaries.images/thread/%3C5b7a6cb0%40news.povray.org%3E/?ttop=428926&toff=150&mtop=4243
> 47
>>
>> --
>> Thomas
> 
> Thanks. A very faithful reproduction of the original, very nice work :-)
> 
>

At the time of building I was surprised about the perspective problems I 
had to tackle. I naively assumed that would be easy but of course the 
lens and its orientation had introduced all kinds of deformations to the 
photograph. Also assuming that walls are straight, as well as streets 
are level... ;-)

-- 
Thomas

Post a reply to this message

Thomas de Groot <tho### [at] degrootorg> wrote:
> Op 5-7-2021 om 05:30 schreef BayashiPascal:
> > Thomas de Groot <tho### [at] degrootorg> wrote:
> >> Op 4-7-2021 om 17:32 schreef Thomas de Groot:
> >>> Op 4-7-2021 om 14:36 schreef BayashiPascal:
> >>>>> 1) You may remember my POV-Ray scene 'Paris la nuit' a couple of years
> >>>>> ago, based on a photograph by Sabine Weiss in 1953. It was just done by
> >>>>> trial and error of course, and a hell of a challenge with all kind of
> >>>>> assumptions. Nothing to do with photogrammetry of course, but I
> >>>>> appreciate your caveats about any "easy magic" ;-).
> >>>>
> >>>> I can't recall it from the name, neither find it with Google. Would
> >>>> you have a
> >>>> link ? I would certainly enjoy seeing it again, as usual with your
> >>>> scenes :-)
> >>>>
> >>> There is this:
> >>>
> >>>
http://news.povray.org/povray.binaries.images/thread/%3C4d8dea6b%40news.povray.org%3E/?mtop=359553
> >>>
> >>>
> >> There is also this. A later version:
> >>
> >>
http://news.povray.org/povray.binaries.images/thread/%3C5b7a6cb0%40news.povray.org%3E/?ttop=428926&toff=150&mtop=4
243
> > 47
> >>
> >> --
> >> Thomas
> >
> > Thanks. A very faithful reproduction of the original, very nice work :-)
> >
> >
>
> At the time of building I was surprised about the perspective problems I
> had to tackle. I naively assumed that would be easy but of course the
> lens and its orientation had introduced all kinds of deformations to the
> photograph. Also assuming that walls are straight, as well as streets
> are level... ;-)
>
> --
> Thomas

Surely !
If you remember my work on Escher's house of stairs, you'll also understand I'm
well aware of the amount of work it must have taken you to achieve this scene.
:-)

Post a reply to this message

Op 06/07/2021 om 05:54 schreef BayashiPascal:
> Thomas de Groot <tho### [at] degrootorg> wrote:
>> Op 5-7-2021 om 05:30 schreef BayashiPascal:
>>> Thomas de Groot <tho### [at] degrootorg> wrote:
>>>> Op 4-7-2021 om 17:32 schreef Thomas de Groot:
>>>>> Op 4-7-2021 om 14:36 schreef BayashiPascal:
>>>>>>> 1) You may remember my POV-Ray scene 'Paris la nuit' a couple of years
>>>>>>> ago, based on a photograph by Sabine Weiss in 1953. It was just done by
>>>>>>> trial and error of course, and a hell of a challenge with all kind of
>>>>>>> assumptions. Nothing to do with photogrammetry of course, but I
>>>>>>> appreciate your caveats about any "easy magic" ;-).
>>>>>>
>>>>>> I can't recall it from the name, neither find it with Google. Would
>>>>>> you have a
>>>>>> link ? I would certainly enjoy seeing it again, as usual with your
>>>>>> scenes :-)
>>>>>>
>>>>> There is this:
>>>>>
>>>>>
http://news.povray.org/povray.binaries.images/thread/%3C4d8dea6b%40news.povray.org%3E/?mtop=359553
>>>>>
>>>>>
>>>> There is also this. A later version:
>>>>
>>>>
http://news.povray.org/povray.binaries.images/thread/%3C5b7a6cb0%40news.povray.org%3E/?ttop=428926&toff=150&mtop=4
> 243
>>> 47
>>>>
>>>> --
>>>> Thomas
>>>
>>> Thanks. A very faithful reproduction of the original, very nice work :-)
>>>
>>>
>>
>> At the time of building I was surprised about the perspective problems I
>> had to tackle. I naively assumed that would be easy but of course the
>> lens and its orientation had introduced all kinds of deformations to the
>> photograph. Also assuming that walls are straight, as well as streets
>> are level... ;-)
>>
>> --
>> Thomas
> 
> Surely !
> If you remember my work on Escher's house of stairs, you'll also understand I'm
> well aware of the amount of work it must have taken you to achieve this scene.
> :-)
> 
Indeed! That is a brilliant piece of work.

-- 
Thomas

Post a reply to this message

"Bald Eagle" <cre### [at] netscapenet> wrote:
> ...

Hi again Bill,
I'll only reply about the paper of Dr Karsch and leave the reply to your other
comments for another (hypothetic) day. Not that they aren't interesting me, not
at all, it's just a time problem...


> a) you could look deeper into his work, with your education and experience in
> the area and see how much of what he claims to accomplish is "embellished"

I've read the paper and had a look at the website of Dr Karsch.
The results introduced in the paper are indeed very good, and certainly not
embellished in any way. You just have to understand the limitations of their
method. It relies heavily on user guidance to provide sufficient information
about the scene geometry, lighting, interaction between the scene and the added
objects, as well as user guidance for correction and supplementation of
automatic estimates. It produces a very coarse representation of the scene
essentially limited to planes, and any more complex object must be modeled with
an external 3D modeling tool and imported, or manually segmented for occlusion
surfaces. It is not applicable to all kind of scenes. It produces qualitatively
good results rather than quantitatively accurate ones.
The only point I would argue with is when they say a novice can with a few
annotations obtain professional results. That's maybe true for cases where their
method works well, but maybe reality is a little more complex for cases where it
doesn't. For example, in equation (1), about the weights they say the "user can
also modify these weights depending on the confidence of their manual source
estimates". This looks to me like needing a bit of expertise to know when and
how to modify these weights. Or, how to split the scene into planes, choose the
type and position of lights, the material of planes (they speak about selecting
reflecting surface for example), ... A simple scene like the one in figure 3 is
surely straight forward, but one like figure 6 or 7 doesn't look so. These are
the kind of things I call 'unspoken and time-consuming pre/post-processing
requiring experience'.
About using their work with POV-Ray, their implementation is done using
LuxRender, so at the very least it would take some refactoring to adapt it to
POV-Ray. It's also basically a method to calculate rendering parameters for
objects a posteriori composited into an existing 2D image. It doesn't seem to
match your expectation: recreating the scene geometry, if I understand well.
Part of their method could be reused, like the calculation of camera parameters
from vanishing points. But rather than deconstruct their work and look for
interesting bits, I would personnally look directly into the relevant papers
about these bits. If their coarse scene made of planes is enough for you, this
could also be reused but there too you would need to deconstruct part of their
work and refactor it to your needs (also it is unclear to me if they really
calculate down to the 3D coordinates of the plane, it seems to me they don't
need it in their method). This may be not easy work, it would discard the most
valuable part of their method, so there again I would probably rather go for my
own implementation from scratch.


> b) if he and his colleagues/coworkers/students have done a lot of the puzzling
> out and heavy lifting already, then perhaps he might provide you with tools that
> would help you in your own work that you might not be familiar with.

About my own work, my goal is quite different (fully automated reconstruction of
highly accurate geometry and texture of a single object from photographs), so
this paper won't help. But it was an interesting read anyway and may be relevant
in a future project. Thank you.


Hope we'll have other opportunities to speak about it in the future! :-)

Pascal

Post a reply to this message

Hi,

> Bald Eagle writes:
>> Take for example Francois LE COAT's very interesting work:
>>
>> http://news.povray.org/povray.advanced-users/thread/%3Cweb.5bb77cec1f3
6de80c437ac910%40news.povray.org%3E/ 
>>
>> Forgive me if there's a lot of things I don't understand, properly rec
all, or
>> have wrong, but it seems to me that an aerial photograph of a landscap
e has a
>> lot of perspective distortion, and part of the photogrammetry process 
would be
>> correcting for that.
> 
> For the time being, I'm experimenting to model trajectories in 3D.
> But it is a difficult work, because public images given by the NASA
> are not of good quality. If the Ingenuity helicopter flying on Mars was

> driven with my trajectory modelling, it would have crashed. But it is
> what I'm measuring, with input data I could collect. This is strange!
> I have no other possibility than to improve my computations further on.


For the 9th flight over Mars, I collected better images from the
surface. To understand what I'm computing, here is another work
from mine...

(1) <https://twitter.com/Astro_Aure/status/1414699163569246210>

You can see a cursor pointing at Ingenuity's localization, moving on
the map, following the helicopter's motion. What I would like to do, is
to compute this in space, with "translate <Tx, Ty, Tz>" in 3 dimensions.

(2) <https://www.youtube.com/watch?v=MT5SqhJ0Jio>

I could also "rotate <Rx, Ry, Rz>" and "shear <Sx, Sy, 0>" but it is not
working for the moment. And I rendered this with Persistence Of Vision.
The algorithm I'm using is probably close to what is used on Mars,
because it is constructed to avoid the drift from the embedded IMU.
It's not viewed from the top (like (1)) but from the surface of Mars.

Best regards,

-- 
François LE COAT
<http://eureka.atari.org/>

Post a reply to this message

Hi,

>> Bald Eagle writes:
>>> Take for example Francois LE COAT's very interesting work:
>>>
>>> http://news.povray.org/povray.advanced-users/thread/%3Cweb.5bb77cec1f
36de80c437ac910%40news.povray.org%3E/ 
>>>
>>> Forgive me if there's a lot of things I don't understand, properly re
call, or
>>> have wrong, but it seems to me that an aerial photograph of a landsca
pe has a
>>> lot of perspective distortion, and part of the photogrammetry process
 would be
>>> correcting for that.
>>
>> For the time being, I'm experimenting to model trajectories in 3D.
>> But it is a difficult work, because public images given by the NASA
>> are not of good quality. If the Ingenuity helicopter flying on Mars wa
s
>> driven with my trajectory modelling, it would have crashed. But it is
>> what I'm measuring, with input data I could collect. This is strange!
>> I have no other possibility than to improve my computations further on
.
> 
> For the 9th flight over Mars, I collected better images from the
> surface. To understand what I'm computing, here is another work
> from mine...
> 
> (1) <https://twitter.com/Astro_Aure/status/1414699163569246210>
> 
> You can see a cursor pointing at Ingenuity's localization, moving on
> the map, following the helicopter's motion. What I would like to do, is

> to compute this in space, with "translate <Tx, Ty, Tz>" in 3 dimensions
.
> 
> (2) <https://www.youtube.com/watch?v=MT5SqhJ0Jio>
> 
> I could also "rotate <Rx, Ry, Rz>" and "shear <Sx, Sy, 0>" but it is no
t
> working for the moment. And I rendered this with Persistence Of Vision.

> The algorithm I'm using is probably close to what is used on Mars,
> because it is constructed to avoid the drift from the embedded IMU.
> It's not viewed from the top (like (1)) but from the surface of Mars.

For 10th flight over Mars, Ingenuity followed an elaborate trajectory...

	<https://www.youtube.com/watch?v=QiF9VJJamkE>

NASA planned to take color pictures of remarkable points on ground.

Best regards,

-- 
François LE COAT
<http://eureka.atari.org/>

Post a reply to this message

Hi,

>>> Bald Eagle writes:
>>>> Take for example Francois LE COAT's very interesting work:
>>>>
>>>> http://news.povray.org/povray.advanced-users/thread/%3Cweb.5bb77cec1
f36de80c437ac910%40news.povray.org%3E/ 
>>>>
>>>> Forgive me if there's a lot of things I don't understand, properly 
>>>> recall, or
>>>> have wrong, but it seems to me that an aerial photograph of a 
>>>> landscape has a
>>>> lot of perspective distortion, and part of the photogrammetry 
>>>> process would be
>>>> correcting for that.
>>>
>>> For the time being, I'm experimenting to model trajectories in 3D.
>>> But it is a difficult work, because public images given by the NASA
>>> are not of good quality. If the Ingenuity helicopter flying on Mars w
as
>>> driven with my trajectory modelling, it would have crashed. But it is

>>> what I'm measuring, with input data I could collect. This is strange!

>>> I have no other possibility than to improve my computations further o
n.
>>
>> For the 9th flight over Mars, I collected better images from the
>> surface. To understand what I'm computing, here is another work
>> from mine...
>>
>> (1) <https://twitter.com/Astro_Aure/status/1414699163569246210>
>>
>> You can see a cursor pointing at Ingenuity's localization, moving on
>> the map, following the helicopter's motion. What I would like to do, i
s
>> to compute this in space, with "translate <Tx, Ty, Tz>" in 3 dimension
s.
>>
>> (2) <https://www.youtube.com/watch?v=MT5SqhJ0Jio>
>>
>> I could also "rotate <Rx, Ry, Rz>" and "shear <Sx, Sy, 0>" but it is n
ot
>> working for the moment. And I rendered this with Persistence Of Vision
.
>> The algorithm I'm using is probably close to what is used on Mars,
>> because it is constructed to avoid the drift from the embedded IMU.
>> It's not viewed from the top (like (1)) but from the surface of Mars.
> 
> For 10th flight over Mars, Ingenuity followed an elaborate trajectory..
.
> 
>      <https://www.youtube.com/watch?v=QiF9VJJamkE>
> 
> NASA planned to take color pictures of remarkable points on ground.

I've been concerned by 11th flight over planet Mars yesterday August
14th, after Ingenuity helicopter happened August 4th...

	<https://www.youtube.com/watch?v=50fccs79W1A>

RAW separated images were released by the NASA at the location:

<https://mars.nasa.gov/mars2020/multimedia/raw-images/?af=HELI_NAV,HELI
_RTE#raw-images>

Then Aurélien Genin put online a MPEG 4 video at the location:

<https://twitter.com/RevesdEspace/status/1426147951693402114>

August 13th. I worked on this video. It takes a delay between the
day of the flight, and final result here. But this video comes
from a very distant location in space!

Best regards,

-- 
François LE COAT
<http://eureka.atari.org/>

Post a reply to this message

Hi,

>>>> Bald Eagle writes:
>>>>> Take for example Francois LE COAT's very interesting work:
>>>>>
>>>>> http://news.povray.org/povray.advanced-users/thread/%3Cweb.5bb77cec
1f36de80c437ac910%40news.povray.org%3E/ 
>>>>>
>>>>> Forgive me if there's a lot of things I don't understand, properly 
recall, or
>>>>> have wrong, but it seems to me that an aerial photograph of a lands
cape has a
>>>>> lot of perspective distortion, and part of the photogrammetry proce
ss would be
>>>>> correcting for that.
>>>>
>>>> For the time being, I'm experimenting to model trajectories in 3D.
>>>> But it is a difficult work, because public images given by the NASA
>>>> are not of good quality. If the Ingenuity helicopter flying on Mars 
was
>>>> driven with my trajectory modelling, it would have crashed. But it i
s
>>>> what I'm measuring, with input data I could collect. This is strange
!
>>>> I have no other possibility than to improve my computations further 
on.
>>>
>>> For the 9th flight over Mars, I collected better images from the
>>> surface. To understand what I'm computing, here is another work
>>> from mine...
>>>
>>> (1) <https://twitter.com/Astro_Aure/status/1414699163569246210>
>>>
>>> You can see a cursor pointing at Ingenuity's localization, moving on
>>> the map, following the helicopter's motion. What I would like to do, 
is
>>> to compute this in space, with "translate <Tx, Ty, Tz>" in 3 dimensio
ns.
>>>
>>> (2) <https://www.youtube.com/watch?v=MT5SqhJ0Jio>
>>>
>>> I could also "rotate <Rx, Ry, Rz>" and "shear <Sx, Sy, 0>" but it is 
not
>>> working for the moment. And I rendered this with Persistence Of Visio
n.
>>> The algorithm I'm using is probably close to what is used on Mars,
>>> because it is constructed to avoid the drift from the embedded IMU.
>>> It's not viewed from the top (like (1)) but from the surface of Mars.

>>
>> For 10th flight over Mars, Ingenuity followed an elaborate trajectory.
..
>>
>>      <https://www.youtube.com/watch?v=QiF9VJJamkE>
>>
>> NASA planned to take color pictures of remarkable points on ground.
> 
> I've been concerned by 11th flight over planet Mars yesterday August
> 14th, after Ingenuity helicopter happened August 4th...
> 
>      <https://www.youtube.com/watch?v=50fccs79W1A>
> 
> RAW separated images were released by the NASA at the location:
> 
> <https://mars.nasa.gov/mars2020/multimedia/raw-images/?af=HELI_NAV,HE
LI_RTE#raw-images> 
> 
> Then Aurélien Genin put online a MPEG 4 video at the location:
> 
> <https://twitter.com/RevesdEspace/status/1426147951693402114>
> 
> August 13th. I worked on this video. It takes a delay between the
> day of the flight, and final result here. But this video comes
> from a very distant location in space!

Aurélien Genin shared rectified images of the 14th flight of Ingenuity:


	<https://twitter.com/Astro_Aure/status/1455284564499341319>

This is interesting because the cadence is higher with 7 images/s. So
reconstructed 3D localization of the helicopter is now more precise...

	<https://www.youtube.com/watch?v=OkclJd3Fmv0>

Ingenuity flew Oct. 24th and the mission is not yet fully accomplished!

Best regards,

-- 
François LE COAT
<http://eureka.atari.org/>

Post a reply to this message

Op 04/07/2021 om 14:42 schreef BayashiPascal:
> Thomas de Groot <tho### [at] degrootorg> wrote:
>> Op 4-7-2021 om 11:09 schreef Mike Horvath:
>>> On 7/4/2021 2:20 AM, Thomas de Groot wrote:
>>>> 2) I have seen on a couple of occasions (on TV), archaeologists make a
>>>> lot of photographs of an object, under all kind of angles, and later
>>>> combine those into a 3d model (with software of course). I saw that
>>
>>>> geology students recently used a drone to photograph the walls of a
>>>> quarry in the same manner, and assembled them into a 3d model of the
>>>> quarry. Fascinating stuff, and relatively cheap to implement,
>>>> especially for students I understood.
>>>>
>>>
>>> I wonder *how* cheap it really is. Probably *not* cheap, in terms of the
>>> work and expertise involved.
>>>
>> I don't really know. What I understood, in particular from the Chinese
>> example, was that the hardware came 'cheap' as only a perfectly common
>> digital camera was needed and no sophisticated laser-controlled stuff.
>> And a simple stepladder in addition, to get around the statues. The same
>> applied for the students with their drone. Concerning the software, I
>> have no idea. The results looked good however, and I doubt that they had
>> any particularly high expertise in the matter.
>>
>> --
>> Thomas
> 
> 
> You can indeed produce good results with a standard camera. The software itself,
> if not freeware is quite expensive, but if they were students it's probably
> provided by their university and that's surely not much compared to a university
> budget. About expertise, if they were students, that very probably mean they had
> teachers with a high level of expertise to guide them.
> 
I agree. I just came across the publication(s) - in Dutch unfortunately 
- I mentioned earlier. It is the software indeed which is (relatively) 
expensive. It makes use of the "Structure from Motion" photogrammetry 
technique. See examples in:

https://sketchfab.com/reindersma


-- 
Thomas

Post a reply to this message