|
 |
I am surely doing something wrong here, any advice or help would be highly
appreciated.
GOAL:
I want to transform the object mesh (.inc) in the same pose as the object in the
captured image and then render it on the object in the captured image.
that is, I would like this rendered object by pov-ray to alight exactly with the
object in the captured image.
--------------------------------------------------------------------------------
DESCRIPTION:
I get my object mesh from the keypose/ BOP dataset, with its corresponding
ground-truth object pose in the form of a Transformation matrix(Original
Transformation matrix.png). This is in the Right Hand Coordinate system.
Since POV-ray operates in the Left-hand coordinate system I tried to convert the
transformation matrix ( the ground-truth pose provided by the dataset) to the
right hand by following suggestions:
https://stackoverflow.com/questions/1263072/changing-a-matrix-from-right-handed-to-left-handed-coordinate-system
That is from the original transformation matrix:
{ rx, ry, rz, 0 }
{ ux, uy, uz, 0 }
{ lx, ly, lz, 0 }
{ px, py, pz, 1 }
To change it from left to right or right to left, flip it like this:
{ rx, rz, ry, 0 }
{ lx, lz, ly, 0 }
{ ux, uz, uy, 0 }
{ px, pz, py, 1 }
But this leads to the following result (image).
Since this throws the object out of the image, i change the "CamLoc" from
"<0,0,0>" to "<0,0,-2>" in the script below (image).
--------------------------------------------------------------------------------
THE SCRIPT I AM USING IS:
#version 3.7;
global_settings {max_trace_level 40 }
#include "colors.inc"
#include "shapes.inc"
#ifdef (cl_x) #declare CamLoc=<0,0,0>; #end // Camera Location
#ifdef (lk_x) #declare LookAt=<0,0,1>; #end // Look at
#ifdef (cs_x) #declare CamSky=<0.0, 1, 0>; #end // Camera Sky
#declare CamDir = vnormalize(LookAt - CamLoc);
#declare CamR = vnormalize(vcross(CamSky,CamDir));
#declare CamUp = vnormalize(vcross(CamDir,CamR));
#ifdef (cam_a) #declare CamA = cam_a; #else #declare CamA = 1; #end // Aspect
ratio
#ifdef (cam_z) #declare CamZ = cam_z; #else #declare CamZ = 1.5; #end // Zoom
#ifdef (bg_sc) #declare BgSc = bg_sc; #else #declare BgSc = 3; #end //
Background scale
#ifndef (IOR) #declare IOR = 1.5; #end // Index of refraction
#ifndef (Trans) #declare Trans = 1; #end // Transmit
#ifndef (Dim) #declare Dim = 1; #end // Ambient intensity
camera {
location CamLoc
direction CamDir * CamZ
angle 86.9
right CamR * CamA
up CamUp
}
#macro OrientZ(p1,p2,cs)
#local nz = vnormalize(p2-p1);
#local nx = vnormalize(vcross(cs,nz));
#local ny = vcross(nz,nx);
matrix <nx.x, nx.y, nx.z,
ny.x, ny.y, ny.z,
nz.x, nz.y, nz.z,
p1.x, p1.y, p1.z>
#end
#ifdef (Calib) // Render objects with graycode patterns
#declare pigmentStructure = pigment {
#if (clock <= 2)
#if (clock = 1) #declare mask = 1; #end color White
#else
image_map { png concat(${ImageName}, str(clock-2,1,0), ".png")
map_type 0 interpolate 2 }
#end
}
#else
#declare pigmentImage = pigment {
#ifdef (mask)
color White
#else
#ifdef (rho) color White #else image_map { jpeg ${ImageName}
map_type 0 interpolate 2 } #end
#end
}
#end
box { <0,0,0> <1,1,0.01>
#ifdef (Calib)
pigment { pigmentStructure }
finish { ambient 1 }
#else
pigment { pigmentImage }
finish { ambient Dim }
#end
translate <-0.5,-0.5,0>
scale <CamA, 1, 1>
translate <0, 0, CamZ>
scale BgSc
OrientZ(CamLoc,LookAt,CamSky)
}
#ifndef (Empty)
object {
shape
#ifdef (mask)
pigment {color Black}
#else
texture {
pigment{
color filter 1
#ifdef (Calib) transmit 1 #else transmit Trans #end
}
}
interior {
ior IOR
#ifndef (Calib)
#ifdef (FadeD) fade_distance FadeD fade_power FadeP #end
#end
}
#end
matrix <0.9985836 , -0.03011384, -0.04386286,
-0.05320521, -0.56519264, -0.82324145,
0. , 0.82440915, -0.56599431,
-0.03461049, 0.7220574 , 0.08761759>
}
#end
PROBLEM:
As you can see this particular transform throws the object out of the image.
Ideally, i would like this rendered object by pov-ray to alight exactly with the
object in the image.
I have also tried a few other transforms but they also do not seem to work.
for reference, i also add the result of the pov-ray rendered image using the
original transform given by the dataset (attached image: )
Any help would be greatly appreciated!
Post a reply to this message
Attachments:
Download 'original transformation matrix.png' (14 KB)
Preview of image 'original transformation matrix.png'
|
|
tuwien
