POV-Ray : Newsgroups : povray.advanced-users : Converting 2D screen coordinates to 3D Server Time 2 Jun 2023 02:10:41 EDT (-0400)
 Converting 2D screen coordinates to 3D (Message 1 to 10 of 20)
 From: Bald Eagle Subject: Converting 2D screen coordinates to 3D Date: 4 Feb 2023 23:25:00 Message:
```
{
"@context": "https://schema.org",
"@type": "DiscussionForumPosting",
"@id": "#web.63df2e78f06dca3d1f9dae3025979125%40news.povray.org",
"headline": "Converting 2D screen coordinates to 3D",
"dateCreated": "2023-02-05T04:25:00+00:00",
"datePublished": "2023-02-05T04:25:00+00:00",
"author": {
"@type": "Person",
"name": "Bald Eagle"
}
}
I'm trying to do a sort of rough photogrammetric conversion of points on the
screen to true 3D coordinates with the same apparent position.

Assume an orthographic camera.
Let's say that I have a line segment extending from the top left of an image to
the lower right.  If I draw a line perpendicular to this, then I can use this
new line as an axis for rotation.

If I rotate a copy of the line segment around this axis, it should visually
remain in line with the original line segment, but the ends would appear to
contract.
If I scaled the copy of the line segment up so that it was the estimated true
size of the original line segment (which represents the view of a longer line
segment extending through the plane of the screen), then when I rotate it the
proper amount, it should exactly line up with the original line segment, but the
endpoints will now have the proper z-coordinates.  We'll call this the z-buffer
axis.

If I take the negative reciprocal of the original line segment's slope, I get a
perpendicular line.  This should correspond to all of the points in the plane
that are of the same 3D z coordinate.  If I rotate this perpendicular line
around the z-buffer axis, it should cross all of the points in the image that
have the same z-value if it were actually in 3D.  We'll call this the xy line.
And if I take the cross product of this line and the z-buffer axis, I should get
the up vector.

If I then take points on the image and apply the same rotational transform to
these as I did to get the z-bufffer axis, and then rotate around that the same
amount as I did to get the xy line, then I should get a coordinate that if
scaled to coincide with its original position will give me it 3D z-coordinate.

Does this make sense, or am I going to be chasing my tail in circles on this?
```
 From: Kenneth Subject: Re: Converting 2D screen coordinates to 3D Date: 6 Feb 2023 08:15:00 Message:
```
{
"@context": "https://schema.org",
"@type": "DiscussionForumPosting",
"@id": "#web.63e0fc80ef540dd79b4924336e066e29%40news.povray.org",
"headline": "Re: Converting 2D screen coordinates to 3D",
"dateCreated": "2023-02-06T13:15:00+00:00",
"datePublished": "2023-02-06T13:15:00+00:00",
"author": {
"@type": "Person",
"name": "Kenneth"
}
}
A couple of initial (naive?) questions, to help me better-visualize the details:
>
> Assume an orthographic camera.
> Let's say that I have a line segment extending from the top left of
> an image to the lower right.

Is that initial line an actual object in the 3-D scene that appears to extend
from corner to corner, or is it a line only in flat 2-D screen space (like
Screen.inc would typically produce, for example)? I'm guessing it's a 3-D
object, extending into the scene.

> If I draw a line perpendicular to this, then I can use this
> new line as an axis for rotation.

Perpendicular to it in x/y screen space, or pointing 'out' from the middle of
the initial line segment directly into the camera? Or maybe pointing somewhere
else in 3-D space? Sorry, this one is hard to visualize.
>
> If I rotate a copy of the line segment around this axis, it should visually
> remain in line with the original line segment, but the ends would appear to
> contract.

I can sort of 'see' this...depending on whether or not I understand the initial
stuff.
```
 From: Bald Eagle Subject: Re: Converting 2D screen coordinates to 3D Date: 6 Feb 2023 13:25:00 Message:
```
{
"@context": "https://schema.org",
"@type": "DiscussionForumPosting",
"@id": "#web.63e1453eef540dd71f9dae3025979125%40news.povray.org",
"headline": "Re: Converting 2D screen coordinates to 3D",
"dateCreated": "2023-02-06T18:25:00+00:00",
"datePublished": "2023-02-06T18:25:00+00:00",
"author": {
"@type": "Person",
"name": "Bald Eagle"
}
}
"Kenneth" <kdw### [at] gmailcom> wrote:
> A couple of initial (naive?) questions, to help me better-visualize the details:
> >
> > Assume an orthographic camera.
> > Let's say that I have a line segment extending from the top left of
> > an image to the lower right.
>
> Is that initial line an actual object in the 3-D scene that appears to extend
> from corner to corner, or is it a line only in flat 2-D screen space (like
> Screen.inc would typically produce, for example)? I'm guessing it's a 3-D
> object, extending into the scene.

That initial line is indeed an axis of the object, which would extend forward
and backward from the image plane if it were an actual 3D object.

> > If I draw a line perpendicular to this, then I can use this
> > new line as an axis for rotation.
>
> Perpendicular to it in x/y screen space, or pointing 'out' from the middle of
> the initial line segment directly into the camera? Or maybe pointing somewhere
> else in 3-D space? Sorry, this one is hard to visualize.

Making the line perpendicular in screen space.  Then when the z-buffer line gets
rotated it remains "in place"/in-line with where it already is, just with +/-
z-components in the vectors of the now-rotated endpoints.

> > If I rotate a copy of the line segment around this axis, it should visually
> > remain in line with the original line segment, but the ends would appear to
> > contract.
>
> I can sort of 'see' this...depending on whether or not I understand the initial
> stuff.

Image of work so far attached.   It's halfway there, though I'm guessing at
magic rotation angle numbers and just flying by the seat of my pants through
intuition.

Apparently there's a much better way to do this that's used in multiple view
geometry in computer vision, but it uses QR decomposition and SVD, among other
matrix operations.   There are Matlab libraries to do some of this, but it would
of course be preferable to be able to do this ourselves in SDL/source.

I should probably see if I can find a copy of Hartley and Zisserman, or the more
modern text that has supplanted it - Photogrammetric Computer Vision:

https://users.cecs.anu.edu.au/~hartley/Papers/CVPR99-tutorial/tut_4up.pdf

https://mathematica.stackexchange.com/questions/9244/solve-system-of-equations-related-to-perspective-projection

https://math.stackexchange.com/questions/242661/finding-a-3d-transformation-matrix-based-on-the-2d-coordinates

https://math.stackexchange.com/questions/455128/how-to-find-an-all-in-one-2d-to-3d-transformation-matrix-for-perspectiv
e-project
```

Attachments:

Preview of image 'yesobjects.png'

 From: yesbird Subject: Re: Converting 2D screen coordinates to 3D Date: 6 Feb 2023 21:00:00 Message:
```
{
"@context": "https://schema.org",
"@type": "DiscussionForumPosting",
"@id": "#web.63e1af9fef540dd780f94ee210800fb2%40news.povray.org",
"headline": "Re: Converting 2D screen coordinates to 3D",
"dateCreated": "2023-02-07T02:00:00+00:00",
"datePublished": "2023-02-07T02:00:00+00:00",
"author": {
"@type": "Person",
"name": "yesbird"
}
}
> Image of work so far attached.   It's halfway there, though I'm guessing at
> magic rotation angle numbers and just flying by the seat of my pants through
> intuition.
Haaa, what I see !
You started 3D reconstruction of Yes-related staff with photogrammetry ...
Very interesting, but it seams to me implementation will be not easy, as we have
only one-view here.

In any case, please tell me if I can help you with some Matlab's methods, you
know, I have one )

>
> https://users.cecs.anu.edu.au/~hartley/Papers/CVPR99-tutorial/tut_4up.pdf
>
>
https://mathematica.stackexchange.com/questions/9244/solve-system-of-equations-related-to-perspective-projection
>
>
https://math.stackexchange.com/questions/242661/finding-a-3d-transformation-matrix-based-on-the-2d-coordinates
>
>
https://math.stackexchange.com/questions/455128/how-to-find-an-all-in-one-2d-to-3d-transformation-matrix-for-perspect
iv
> e-project

I am completely new to photogrammetry, this is a good starting point, thanks.
```
 From: yesbird Subject: Re: Converting 2D screen coordinates to 3D Date: 6 Feb 2023 21:10:00 Message:
```
{
"@context": "https://schema.org",
"@type": "DiscussionForumPosting",
"@id": "#web.63e1b247ef540dd780f94ee210800fb2%40news.povray.org",
"headline": "Re: Converting 2D screen coordinates to 3D",
"dateCreated": "2023-02-07T02:10:00+00:00",
"datePublished": "2023-02-07T02:10:00+00:00",
"author": {
"@type": "Person",
"name": "yesbird"
}
}
Also I suppose that R (https://www.r-project.org/) should have some useful
methods we can use for experiments, before SDL implementation.
```
 From: yesbird Subject: Re: Converting 2D screen coordinates to 3D Date: 6 Feb 2023 21:25:00 Message:
```
{
"@context": "https://schema.org",
"@type": "DiscussionForumPosting",
"@id": "#web.63e1b66bef540dd780f94ee210800fb2%40news.povray.org",
"headline": "Re: Converting 2D screen coordinates to 3D",
"dateCreated": "2023-02-07T02:25:00+00:00",
"datePublished": "2023-02-07T02:25:00+00:00",
"author": {
"@type": "Person",
"name": "yesbird"
}
}
Here is a list of all R packages:
https://cran.r-project.org/web/packages/available_packages_by_name.html
```
 From: Bald Eagle Subject: Re: Converting 2D screen coordinates to 3D Date: 7 Feb 2023 06:25:00 Message:
```
{
"@context": "https://schema.org",
"@type": "DiscussionForumPosting",
"@id": "#web.63e234beef540dd71f9dae3025979125%40news.povray.org",
"headline": "Re: Converting 2D screen coordinates to 3D",
"dateCreated": "2023-02-07T11:25:00+00:00",
"datePublished": "2023-02-07T11:25:00+00:00",
"author": {
"@type": "Person",
"name": "Bald Eagle"
}
}
"yesbird" <nomail@nomail> wrote:

> Haaa, what I see !
> You started 3D reconstruction of Yes-related staff with photogrammetry ...
> Very interesting, but it seams to me implementation will be not easy, as we have
> only one-view here.

Yes, I would imagine that this merely sets us up for a situation much like
solving for acos (theta) - there are an infinite number of solutions, however
only one or two may make any sense, and we usually just go with the minimum
positive solution.

> I am completely new to photogrammetry, this is a good starting point, thanks.

How does this look?
https://www.robots.ox.ac.uk/~vgg/hzbook/code/

Also, I think that Francois LeCoat might have a few tricks up his sleeve - he
seems to have been doing some pretty sharp work these past several years.

- BE
```
 From: Bald Eagle Subject: Re: Converting 2D screen coordinates to 3D Date: 7 Feb 2023 07:10:00 Message:
```
{
"@context": "https://schema.org",
"@type": "DiscussionForumPosting",
"@id": "#web.63e23ec7ef540dd71f9dae3025979125%40news.povray.org",
"headline": "Re: Converting 2D screen coordinates to 3D",
"dateCreated": "2023-02-07T12:10:00+00:00",
"datePublished": "2023-02-07T12:10:00+00:00",
"author": {
"@type": "Person",
"name": "Bald Eagle"
}
}
I wanted to do something a bit more challenging than the floating rocks and
islands (but super interested in getting a bunch of those going)
Of course "a bit" tends to get me deep into the weeds at every turn.

He also drew/painted an amazing cloud formation on
Like it is  YES   at he Mesa Arts Center 2 CD & DVD set
that reminded of some silly clouds I made way back:

http://news.povray.org/povray.binaries.images/message/%3Cweb.592d5ce4fb337a11c437ac910%40news.povray.org%3E/#%3Cweb.592
d5ce4fb337a11c437ac910%40news.povray.org%3E

Also, the amazing cloud/storm models made by Leigh Orf in the PDF that you
referenced.   (Also, that df3 brain is from Paul Bourke)
```
 From: yesbird Subject: Re: Converting 2D screen coordinates to 3D Date: 7 Feb 2023 07:55:00 Message:
```
{
"@context": "https://schema.org",
"@type": "DiscussionForumPosting",
"@id": "#web.63e249b4ef540dd7164c19e610800fb2%40news.povray.org",
"headline": "Re: Converting 2D screen coordinates to 3D",
"dateCreated": "2023-02-07T12:55:00+00:00",
"datePublished": "2023-02-07T12:55:00+00:00",
"author": {
"@type": "Person",
"name": "yesbird"
}
}
> How does this look?
> https://www.robots.ox.ac.uk/~vgg/hzbook/code/

Very promising ! Will examine it when find a time, this will be a serious
investigation.
```
 From: yesbird Subject: Re: Converting 2D screen coordinates to 3D Date: 7 Feb 2023 08:10:00 Message:
```
{
"@context": "https://schema.org",
"@type": "DiscussionForumPosting",
"@id": "#web.63e24c64ef540dd7164c19e610800fb2%40news.povray.org",
"headline": "Re: Converting 2D screen coordinates to 3D",
"dateCreated": "2023-02-07T13:10:00+00:00",
"datePublished": "2023-02-07T13:10:00+00:00",
"author": {
"@type": "Person",
"name": "yesbird"
}
}
"Bald Eagle" <cre### [at] netscapenet> wrote:
> I wanted to do something a bit more challenging than the floating rocks and
> islands (but super interested in getting a bunch of those going)
> Of course "a bit" tends to get me deep into the weeds at every turn.

Don't you think about using some external tools, just for reference and
experiments ? Yesterday I googled about photogrammetry and found some popular
open-source, like: https://micmac.ensg.eu/index.php/Accueil

>
> He also drew/painted an amazing cloud formation on
> Like it is  YES   at he Mesa Arts Center 2 CD & DVD set

These clouds ?
:)))

> that reminded of some silly clouds I made way back:
>
http://news.povray.org/povray.binaries.images/message/%3Cweb.592d5ce4fb337a11c437ac910%40news.povray.org%3E/#%3Cweb.5
92
> d5ce4fb337a11c437ac910%40news.povray.org%3E

That reminds me some Dali's paintings: