robotica - home.deib.polimi.ithome.deib.polimi.it/restelli/mywebsite/pdf/lezione2.pdf · 13/10/2006...
TRANSCRIPT
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
22
TodayToday
● The last episode ● A survey on Geometry Trasformations ● More useful information about Cameras● How to calibrate a camera
– Example from Bouguet Calibration Toolbox
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
33
The last episodeThe last episode
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
44
Modeling projectionModeling projection
Projection equations– Compute intersection with PP of ray from (x,y,z) to COP– Derived using similar triangles (on board)
– Ignoring the final image coordinate: (from 3D to 2D)
x= fXZ
y= f YZ
X ,Y , Z T f XZ, fYZT
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
55
Principal Point OffsetPrincipal Point Offset
X ,Y , Z T f XZ px , f
YZ p y
T
● The origin oh coordinates in image planes is not at the principal point p x , p yT
XYZ1 fXZpx
fYZp yZ =[ f 0 px 0
0 f p y 00 0 1 0 ] XYZ1
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
66
Camera Calibration Matrix (K)Camera Calibration Matrix (K)
K=[ f 0 px0 f p y0 0 1 ]
x=K [ I∣0] X cam● Rewriting the equation:
● The camera is assumed to be located at the origin of Euclidean coordinate system, with principal axis of camera pointing down the Z-axis: this system may be called camera coordinate frame.
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
77
Camera Rotation and TraslationCamera Rotation and Traslation
X cam=R X− C is an inhomogeneus point in world coordinate frame
represents the same point in camera coordiante frame
represents the coordinate of the camera centre in world coordinate frame
X cam=[R −R C0 1 ][ XY1 ]
x=KR [ I∣− C ] X
X
X cam
C
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
88
Camera Rotation and TraslationCamera Rotation and Traslation
● It is often convenient not to make the camera centre explicit:
X cam=R Xt t=−R C
P=K [R∣t ] Camera MatrixCamera Matrix
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
99
TrasformationsTrasformations
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
1010
Projective TrasformationsProjective Trasformations
A projectivity is an invertible mapping h from P2 to itself such that three points x1,x2,x3 lie on the same line if and only if h(x1),h(x2),h(x3) do.
Definition:
A mapping h:P2→P2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P2 reprented by a vector x it is true that h(x)=Hx
Theorem:
Definition: Projective transformation
=
3
2
1
333231
232221
131211
3
2
1
'
'
'
x
x
x
hhh
hhh
hhh
x
x
x
xx' H=or
projectivity=collineation=projective transformation=homography
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
1111
Mapping between planesMapping between planes
central projection may be expressed by x’=Hx(application of theorem)
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
1212
RRemoving projective distortionemoving projective distortion
333231
131211
3
1
'
''
hyhxh
hyhxh
x
xx
++++==
333231
232221
3
2
'
''
hyhxh
hyhxh
x
xy
++++==
( ) 131211333231' hyhxhhyhxhx ++=++( ) 232221333231' hyhxhhyhxhy ++=++
select four points in a plane with know coordinates
(linear in hij)
(2 constraints/point, 8DOF ⇒ 4 points needed)
Remark: no calibration at all necessarybetter ways to compute
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
1313
A hierarchy of trasformationsA hierarchy of trasformations
• Projective linear group
• Affine group (last row (0,0,1))
• Euclidean group (upper left 2x2 orthogonal)
– Oriented Euclidean group (upper left 2x2 det 1)
• Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant
e.g. Euclidean transformations (traslation and rotation) leave distances unchanged
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
1414
Class I: IsometriesClass I: Isometries
● Trasformation of the plane R2 that preserve Euclidean distance (iso = same, metric= measure )
● A planar Euclidean trasformation can be written:
−=
1100
cossin
sincos
1
'
'
y
x
t
t
y
x
y
x
θθεθθε
1=ε1−=ε
Orientation preserving (Euclidean trasformation):
Orientation reversing:
x0
xx'
==
1
tT
RHE IRR =T
Invariants: length, angle, area
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
1515
Class II: SimilarityClass II: Similarity
● An isometry composed with an isotropic scaling
● And in block form:
−=
1100
cossin
sincos
1
'
'
y
x
tss
tss
y
x
y
x
θθθθ
x0
xx'
==
1
tT
RH
sS IRR =T
Invariants: ratios of length, angle, ratios of areas, parallel lines
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
1616
Class III: Affine tran.Class III: Affine tran.
● A non-singular linear trasformation followed by a traslation
● And in block form:
=
11001
'
'
2221
1211
y
x
taa
taa
y
x
y
x
x0
xx'
==
1
tT
AH A
Invariants: parallel lines, ratios of parallel lengths, ratios of areas
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
1717
Class IV: Projective tras.Class IV: Projective tras.
● A general non-singular linear trasformation of homogeneous coordinate
xv
xx'
==
vP T
tAH ( ) T
21,v vv=
Invariants: cross-ratio of four points on a line (ratio of ratio)
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
1818
What is Cross-Ratio?What is Cross-Ratio?
● 1D case:
x'x 22×=H( ) T21, xx
=
22
11detx,x
ji
jiji xx
xxCrossx1,x2,x3,x4=∣x1,x3∣∣x2,x4∣∣x2,x3∣∣x1,x4∣
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
1919
The line at infinityThe line at infinity
● Affine trasformation:
● Projective trasformation:
=
000 2
1
2
1
x
xx
x
vAA
T
t
+
=
2211
2
1
2
1
0v
xvxvx
xx
x
vAA
T
t
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
2020
Overview transformationsOverview transformations
1002221
1211
y
x
taa
taa
1002221
1211
y
x
tsrsr
tsrsr
333231
232221
131211
hhh
hhh
hhh
1002221
1211
y
x
trr
trr
Projective8dof
Affine6dof
Similarity4dof
Euclidean3dof
Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio
Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l∞
Ratios of lengths, angles.The circular points I,J
lengths, areas.
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
2121
Overview transformations in Overview transformations in 3D3D
10
tRT
s
Projective15dof
Affine12dof
Similarity7dof
Euclidean6dof
Intersection and tangency
Parallellism of planes,Volume ratios, centroids,The plane at infinity π∞
The absolute conic Ω∞
Volume
10
tRT
[A t0T 1]
[A tV T v ]
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
2222
More useful information about Cameras
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
2323
CCD CamerasCCD Cameras
● The pinhole camera model assumes that image coordinate Euclidean– Equal scales in both axial direction
● In CCD there is the possibility of non-square pixels
● Image coordinates measured in pixel introduces unequal scale factors in each direction
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
2424
CCD CamerasCCD Cameras
● If the number of pixels per unit distance coordinates are mx and my in the x and y direction:
● Where:
– e represent the focal length of camera in term of pixel dimensions
– is the principal point in terms of pizel dimensions with coordinates
K=[x 0 x00 y y00 0 1 ]
x= f mx y= f m y
x0= x0 , y0x0=mx px y0=m y p y
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
2525
Finite Projective cameraFinite Projective camera
● For added generality:
● Where s is referred to as the skew parameter– Usually is zero for most normal cameras
K=[x s x00 y y00 0 1 ]
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
2626
Finite Projective cameraFinite Projective camera
● A camera for which the calibration matrix K is of the previous form will be called a finite projective camera
● Notes:– The matrix KR is 3x3 non-singular (a signature)
– Usually you have P so decomposition is needed
P=KR [ I∣− C ]
P=M [ I∣M−1 p4]=KR [ I∣− C ]P=[ p11 p12 p13 p14p21 p22 p23 p24p31 p32 p33 p34
]
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
2727
Camera anatomyCamera anatomy
•Camera center
•Column points
•Principal plane
•Axis plane
•Principal point
•Principal ray
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
2828
CameraCamera centercenter
● Do you remember?
● Then
P=M [ I∣M−1 p4]=KR [ I∣− C ]
C= CT ,1T
C=−M−1 p41
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
2929
Column VectorsColumn Vectors
● Vanishing points of the world coordinate X,Y,Z
p1=PD x
Dx=1000
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
3030
Row VectorsRow Vectors
● The row of the projective camera are 4-vectors which may be interpreted geometrically as particular world plane
P=[ p11 p12 p13 p14p21 p22 p23 p24p31 p32 p33 p34
]=[P1T
P2T
P3T]
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
3131
Principal PlanePrincipal Plane
● The plane through the camera centre parallel to the image plane
It consists of the set of point X which are imaged on the line at infinity of the image
=
1p
p
p
0 3
2
1
Z
Y
X
y
x
T
T
T
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
3232
Axis PlaneAxis Plane
● Points that satisfy or
=
1p
p
p0
3
2
1
Z
Y
X
w
yT
T
T
P1T X=0 P2T X=0
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
3333
The principal pointThe principal point
● The principal axis is the line passing through the camera centre C with direction perpendicular to the principal plane P3 (the axis intersects the image plane at principal point)
principal point
( )0,,,p̂ 3332313 ppp=
∞
330 Mmp̂Px ==P=[M∣p4]
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
3434
Camera CalibrationCamera Calibration
● Common approach
Planar patternNon-planar pattern
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
3535
Camera Calibration from Camera Calibration from Planar PatternsPlanar Patterns
● ICCV Zhang’99: “Flexible Calibration by Viewing a Plane From Unknown Orientations”
m = [Camera Projection Matrix] M
A [R t]
observedestimate: A [R t] M
Minimize:
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
3636
Camera Calibration from Camera Calibration from Planar PatternsPlanar Patterns
● ICCV Zhang’99: “Flexible Calibration by Viewing a Plane From Unknown Orientations”
m = [Camera Projection Matrix] M
A [R t]
observedestimate: A [R t] M
Minimize:
• Two steps:• Find an initial solution for A [R t]• Minimize the objective function
using the initial solution
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
3737
Camera Calibration from Camera Calibration from Planar PatternsPlanar Patterns
• Finding an initial solution– First step
• Estimate the image homography matrix H for each image– Second step
• Solve for b in the linear system:• b yields the intrinsic parameter matrix A.
V b = 0
13/10/200613/10/2006 Corso di Robotica 06/07Corso di Robotica 06/07Davide Migliore - [email protected] Migliore - [email protected]
3838
Camera Calibration from Camera Calibration from Planar PatternsPlanar Patterns
● Intrinsic Parameters:
● Extrinsic Parameters