Hi everyone!

I've been away from the newsgroups for awhile (yet again), but I certainly
haven't been away from POV-Ray. I've been using it almost daily, working on two
long-standing personal projects.

First, I would like to add my congratulations and thanks to the POV team for
releasing v3.7! I can only imagine all of the hard work that went into that. I
haven't played with it much yet, as my two projects were developed in 3.62 and
were keeping me busy;  but I'm looking forward to discovering all the
new/improved features that have been added.

BTW, I recently retired (it's called 'early retirement' in the US.) So now I
have even MORE time to spend with POV-Ray ;-)

By far, the most interesting project I've been developing is 'matchmoving' (also
called camera tracking.) That is, taking a digital video and getting POV-Ray to
'match' the camera movement, while rendering CG objects for later inclusion
(compositing) into the video scene. My process makes use of various free apps:
VirtualDub (with a *magical* motion-analysis plug-in called Deshaker, by Gunnar
Thalin); SMPlayer (a GUI version of MPlayer) for converting my h.264-encoded
videos into individual frames; Edit Pad Lite, for editing the Deshaker data file
into a form that can be #read by POV-Ray; and a modified version of POV-Ray's
"Screen.inc" include file.

The Deshaker plug-in is not meant for 'matchmoving' per se; it's for smoothing
out bumps and shakes in a video. But the data file it generates (for its own use
in VirtualDub) is quite robust, and can instead be used in POV-Ray for
matchmoving, as I've discovered. The method to translate that raw data into
meaningful POV-Ray object/camera transformations took quite awhile to figure
out!

The matchmoving is done in POV-Ray, while rendering the CGI scene objects; the
camera there rotates in all axes to match the pan/tilt/roll movements in the
video, frame by frame. (My code can even match *spatial* 3D movement of the
camera-- with a fair amount of manual help.) Currently, my process is about
2/3rds automated and 1/3rd hard manual labor (only when *spatial* movement needs
matching, as in walking forward with the camera.) If the camera is stationary in
space-- with only pan/tilt/roll-- it's almost fully automated. The animation
scene file(s) to do this are probably the most complex code I've yet written,
and I continue to tweak it all.

The only current limitation of the process (and it's a big one) is that the
POV-Ray camera needs to remain at the origin ( <0,0,0> ). Thus, all the CGI
objects need to move instead.

I'm currently working on a way to automate the *spatial* matchmoving as well, by
detecting 'marker objects' in the video and using that data to create 3D
movements of the POV-Ray camera (or objects), with the correct perspective and
vanishing point. (Deshaker is no help with that.) It's a tricky procedure, as
the data originates from 2D 'planar' images!

This is just a brief description; more details to come.

The results look really promising so far; I will post a recent animation test at
p.b.animations. Meanwhile, here's a rather non-informative still frame ;-)

Post a reply to this message

Attachments:
Download 'matchmoving_still_frame.jpg' (252 KB)

Preview of image 'matchmoving_still_frame.jpg'