Hi everyone!

I've been away from the newsgroups for awhile (yet again), but I certainly
haven't been away from POV-Ray. I've been using it almost daily, working on two
long-standing personal projects.

First, I would like to add my congratulations and thanks to the POV team for
releasing v3.7! I can only imagine all of the hard work that went into that. I
haven't played with it much yet, as my two projects were developed in 3.62 and
were keeping me busy;  but I'm looking forward to discovering all the
new/improved features that have been added.

BTW, I recently retired (it's called 'early retirement' in the US.) So now I
have even MORE time to spend with POV-Ray ;-)

By far, the most interesting project I've been developing is 'matchmoving' (also
called camera tracking.) That is, taking a digital video and getting POV-Ray to
'match' the camera movement, while rendering CG objects for later inclusion
(compositing) into the video scene. My process makes use of various free apps:
VirtualDub (with a *magical* motion-analysis plug-in called Deshaker, by Gunnar
Thalin); SMPlayer (a GUI version of MPlayer) for converting my h.264-encoded
videos into individual frames; Edit Pad Lite, for editing the Deshaker data file
into a form that can be #read by POV-Ray; and a modified version of POV-Ray's
"Screen.inc" include file.

The Deshaker plug-in is not meant for 'matchmoving' per se; it's for smoothing
out bumps and shakes in a video. But the data file it generates (for its own use
in VirtualDub) is quite robust, and can instead be used in POV-Ray for
matchmoving, as I've discovered. The method to translate that raw data into
meaningful POV-Ray object/camera transformations took quite awhile to figure
out!

The matchmoving is done in POV-Ray, while rendering the CGI scene objects; the
camera there rotates in all axes to match the pan/tilt/roll movements in the
video, frame by frame. (My code can even match *spatial* 3D movement of the
camera-- with a fair amount of manual help.) Currently, my process is about
2/3rds automated and 1/3rd hard manual labor (only when *spatial* movement needs
matching, as in walking forward with the camera.) If the camera is stationary in
space-- with only pan/tilt/roll-- it's almost fully automated. The animation
scene file(s) to do this are probably the most complex code I've yet written,
and I continue to tweak it all.

The only current limitation of the process (and it's a big one) is that the
POV-Ray camera needs to remain at the origin ( <0,0,0> ). Thus, all the CGI
objects need to move instead.

I'm currently working on a way to automate the *spatial* matchmoving as well, by
detecting 'marker objects' in the video and using that data to create 3D
movements of the POV-Ray camera (or objects), with the correct perspective and
vanishing point. (Deshaker is no help with that.) It's a tricky procedure, as
the data originates from 2D 'planar' images!

This is just a brief description; more details to come.

The results look really promising so far; I will post a recent animation test at
p.b.animations. Meanwhile, here's a rather non-informative still frame ;-)

Post a reply to this message

Attachments:
Download 'matchmoving_still_frame.jpg' (252 KB)

Preview of image 'matchmoving_still_frame.jpg'

Wow! That's awesome!

Post a reply to this message

On Fri, 14 Feb 2014 00:37:11 +0200, Kenneth <kdw### [at] gmailcom> wrote:

This is awesome!

I wonder how the mobile app FXGuru does the tracking with camera movement.

BTW on this scene - how about using the background photo as a sky_sphere  
so the reflections match the scenery a bit more?


-- 
-Nekar Xenos-

Post a reply to this message

"jhu" <nomail@nomail> wrote:
> Wow! That's awesome!

Yes,

Were you aware that Blender had matchmoving features?

Maybe some pieces of the code might be of some help to your project?

Post a reply to this message

That is looking great indeed. So, not only early retirement (congrats!) 
but you now own a first generation new concept car! Some guys have it 
all... :-)

I second Nekar's suggestion.

Thomas

Post a reply to this message

"jhu" <nomail@nomail> wrote:
> Wow! That's awesome!

Thank you!

Post a reply to this message

"Nekar Xenos" <nek### [at] gmailcom> wrote:
> On Fri, 14 Feb 2014 00:37:11 +0200, Kenneth <kdw### [at] gmailcom> wrote:
>
> This is awesome!

Thanks!
>
> I wonder how the mobile app FXGuru does the tracking with camera movement.

Hmm, that's new to me; I'll have to look it up. Sounds interesting.
>
> BTW on this scene - how about using the background photo as a sky_sphere
> so the reflections match the scenery a bit more?

I gave that some thought; but the video image background has such a limited
field of view that I thought it might look strange as a 'spherical' reflection
(i.e., mapped to a sphere.) The problem with using a sky_sphere is that my
police car renders need a transparent background (for compositing later); I
don't *think* a sky_sphere can be made no_image (?). But I could be wrong about
that.

The reflections do need better imagery, though. I also thought of going out in
my driveway and making some stitched-together photos of the surroundings, from
the 'location' of the police car. But my original background movie was made last
Spring, when all the trees etc. were in bloom; now it's winter and the trees are
bare! :-/

Post a reply to this message

Thomas de Groot <tho### [at] degrootorg> wrote:
> That is looking great indeed. So, not only early retirement (congrats!)
> but you now own a first generation new concept car! Some guys have it
> all... :-)
>
> I second Nekar's suggestion.
>

Thanks, Thomas. Yeah, with all the snow that has been falling here in Virginia
this winter, it would be NICE to have a flying car to escape it all!

Post a reply to this message

"Mr" <nomail@nomail> wrote:
> "jhu" <nomail@nomail> wrote:
> > Wow! That's awesome!
>
> Yes,
>
> Were you aware that Blender had matchmoving features?
>
> Maybe some pieces of the code might be of some help to your project?

I did read that somewhere. What a nice feature! I wonder how expert it is in
determining *spatial* movement of the video camera?

There's another (dedicated) matchmoving program called MOCHA that I would love
to have. But then, it would remove the fun of trying to code my own version in
POV-Ray! ;-)

Post a reply to this message

On 14/02/2014 5:02 PM, Kenneth wrote:
> I
> don't*think*  a sky_sphere can be made no_image (?). But I could be wrong about
> that.

You could always use a large sphere with a spherical projection and the 
no image flag.


-- 
Regards
     Stephen

Post a reply to this message