Hi,

Bald Eagle writes:
> Francois LE COAT wrote:
>> It is possible to perceive the relief (in depth) of a scene, when we
>> have at least two different viewpoints of it. Here is a new example wi
th
>> a drone flying in the middle of a forest of trees, and from which we
>> process the video stream from the embedded camera...
>>
>> <https://www.youtube.com/watch?v=WJ20EBM3PTc>
>>
>> When the two views of the same scene are distant in space, we speak
>> of "spatial disparity". In the present case, the two viewpoints are
>> distant in time, and we then speak of "temporal disparity". This
>> involves knowing whether the two images of the same scene are acquired

>> simultaneously, or delayed in time. We can perceive the relief in dept
h
>> in this case, with a single camera and its continuous video stream.
> 
> Francois,
> 
> Although I believe that I understand the general idea of what you're do
ing in
> your work, it's a bit difficult to fully grasp the details from the vid
eo.
> 
> I'm assuming that you're taking the first frame as a reference image, a
nd then
> reorienting the second frame to optimize the registration.
> Then you're using the OpenCV to do some sort of photogrammetry to gener
ate a
> projection matrix from the 2D image, and extract/back-calculate 3D data
 from
> that matrix for everything in the frame.
> 
> Then you move to the pair of 2nd frame + 3rd frame and repeat the proce
ss.
> 
> Is this correct?

I have the first reference image, and the second one, from which I
determine the optical-flow. That means every pixel integer displacement
between an image, to the other. That gives a vector field that can
be approximated by a global projective transformation. That means eight
parameters in rotation and translation, that match the two images the
best. The quality of the images' matching is measured with correlation,
between the first image and the second (projectively) transformed.

I repeat the treatments with the third, the fourth that gives me a
decreasing correlation value... Until correlation drops below 60%.

Then when correlation is too low, I take a new reference image (the
last good matching) let's admit fourth, and continue to determine
good matching (>60% correlation) with fifth, sixth, and so forth ...

That gives me reliable 8 projective transform parameters between 1st
and 4th images, because of the good correlation. That also gives me
the reliable vector field of the optical-flow, that can be named
"temporal disparity" horizontal and vertical. That's what you're seeing
in grey levels images.

Disparity field which is an integer displacement between images, is
linked to depth in the image (3rd dimension) with an inverse relation
(depth=base/disparity). That means we can evaluate image's depth from
a continuous video stream.

> I'm also wondering if you could create a 3D rendering with the data you
're
> extracting, and maybe a 2D orthographic overhead map of the scene that 
the
> drones are flying through, mapping the position of the drones in the fo
rest.

The disparity measurements are a physical approximation (optical
flow) that is imperfect, and is constantly improved with science
researches. I'm always interested in the progress with this domain.
But for the moment we must have a compromise between quality and
computing time. All is not so perfect, that we can imagine what
you're wishing, from the state of the art...

> As always, I am very interested in understanding the details of what yo
u're
> doing.

I hope you understood it better. I may be a little confuse...

> This is great work, and I hope you are properly recognized for your
> achievements.
> 
> - BW

It's a long work since I can show some promising results. Now I'm
happy to share it with a larger audience. Unfortunately all the
good people that worked with me, are not here to appreciate. That's
why I'm surprised with your interest =)

Thanks for your attention.

Best regards,

-- 

<https://eureka.atari.org/>

Post a reply to this message