image transforms 主講人:虞台文. content overview convolution edge detection – gradients...
TRANSCRIPT
Image Transforms
主講人:虞台文
Content Overview Convolution Edge Detection
– Gradients– Sobel operator– Canny edge detector– Laplacian
Hough Transforms Geometric Transforms
– Affine Transform– Perspective Transform
Histogram Equalization
Image Transforms
Overview
Image Transform Concept
T[]T[]
Image Transform Concept
T[]T[]
( , )I x y( , )I x y ( , )O x y( , )O x y
( , ) ( , )TO x y I x y
Image Transforms
Convolution
Image Convolution
( , )g x y
( , )f x y ( * )( , )f g x y*
g(x,y) is known as convolution kernel.
( * )( , ) ( , ) ( , )v u
f g x y f u v g x u y v
Image Convolution
( , )g x y
( , )f x y ( * )( , )f g x y*
g(x,y) is known as convolution kernel.
( * )( , ) ( , ) ( , )v u
f g x y f u v g x u y v
( * )( , ) ( , ) ( , )y h x w
v y h u x w
f g x y f u v g x u y v
height 2h + 1width 2w + 1
Image Convolution
g(x,y) is known as convolution kernel.
( * )( , ) ( , ) ( , )y h x w
v y h u x w
f g x y f u v g x u y v
height 2h + 1width 2w + 1
Some Convolution Kernels
OpenCV Implementation Image Filter
void cvFilter2D( const CvArr* src, CvArr* dst, const CvMat* kernel, CvPoint anchor=cvPoint(-1, -1));
void cvFilter2D( const CvArr* src, CvArr* dst, const CvMat* kernel, CvPoint anchor=cvPoint(-1, -1));
Deal with Convolution Boundaries
void cvCopyMakeBorder( const CvArr* src, CvArr* dst, CvPoint offset, int bordertype, CvScalar value=cvScalarAll(0));
void cvCopyMakeBorder( const CvArr* src, CvArr* dst, CvPoint offset, int bordertype, CvScalar value=cvScalarAll(0));
Image Transforms
Edge Detection
Edge Detection
Convert a 2D image into a set of curves– Extracts salient features of the scene– More compact than pixels
Origin of Edges
depth discontinuity
surface color discontinuity
illumination discontinuity
surface normal discontinuity
Edges are caused by a variety of factors
Edge Detection
How can you tell that a pixel is on an edge?
Edge Types
Step Edges
Roof Edge Line Edges
Real Edges
Noisy and Discrete!
x
I
We want an Edge Operator that produces:– Edge Magnitude– Edge Orientation– High Detection Rate and Good Localization
Derivatives of Image in 1D
( )I x
2 ( ) ( )I x I x
( )I I x
Edges can be characterized as either:– local extrema of I(x)– zero-crossings of 2I(x)
1D image
gradient
Laplacian
2D-Image Gradient
( , ) ,T
I II x y
x y
,T
x yI I
2D-Image Gradient ( , ) , ,
TT
x y
I II x y I I
x y
Gives the direction of most rapid change in intensity
Gradient direction:
Edge strength:
,0T
xI I
0,T
yI I
,T
x yI I I
1tan /y xI I
2 2x yI I I
Classification of Points
To precisely locate the
edge, we need to thin.
Ideally, edges should be
only one point thick.
TI
Non-zeroedge width
I T
( , ) , ,T
T
x y
I II x y I I
x y
The Sobel Operators
( , ) , ,T
T
x y
I II x y I I
x y
-1 0 1
-2 0 2
-1 0 1
1 2 1
0 0 0
-1 -2 -1
x xI S y yI S
Sobel (3 x 3):
Sobel (5 x 5):
-1 -2 0 2 1
-2 -3 0 3 2
-3 -5 0 5 3
-2 -3 0 3 2
-1 -2 0 2 1
1 2 3 2 1
2 3 5 3 2
0 0 0 0 0
-2 -3 -5 -3 -2
-1 -2 -3 -2 -1
Good LocalizationNoise SensitivePoor Detection
Poor LocalizationLess Noise SensitiveGood Detection
OpenCV Implementation The Sobel Operators
( , ) , ,T
T
x y
I II x y I I
x y
void cvSobel( const CvArr* src, CvArr* dst, int xorder, int yorder, int aperture_size = 3);
void cvSobel( const CvArr* src, CvArr* dst, int xorder, int yorder, int aperture_size = 3);
OpenCV Implementation The Scnarr Operator
( , ) , ,T
T
x y
I II x y I I
x y
void cvSobel( const CvArr* src, CvArr* dst, int xorder, int yorder, int aperture_size = 3);
void cvSobel( const CvArr* src, CvArr* dst, int xorder, int yorder, int aperture_size = 3);
aperture_size
CV_SCHARR
Demonstration
( , )I x y
( , ) , ,T
T
x y
I II x y I I
x y
xI
yI2 2x yI I
Exercise
DownloadTest Program
DownloadTest Program
Effects of Noise
Where is the edge?Where is the edge?
Consider a single row or column of the image
Solution: Smooth First
f
g
*g f
*x g f
Solution: Smooth First
f
g
*g f
*x g f
Where is the edge?Where is the edge?
Derivative Theorem of Convolution
( , )g x y I *I g
2 2
2 2
1( , ) exp
2 2
x yg x y
Gaussian:
( , ) ( , )h x y g x y
2 2
4 2
2 2
4 2
( , )exp
2 2
( , )exp
2 2
g x y x x y
x
g x y y x y
y
Derivative Theorem of Convolution
saves us one operation. * *x xg f g f
x g
f
*x g f
Optimal Edge Detection: Canny
Assume: – Linear filtering– Additive iid Gaussian noise
An "optimal" edge detector should have:– Good Detection
Filter responds to edge, not noise.– Good Localization
detected edge near true edge.– Single Response
one per edge.
Optimal Edge Detection: Canny
Based on the first derivative of a Gaussian
Detection/Localization trade-off– More smoothing improves detection– And hurts localization.
Stages of the Canny algorithm
Noise reductionSize of Gaussian filter
Finding the intensity gradient of the image Non-maximum suppression Tracing edges through the image and hyste
resis thresholdingHigh thresholdLow threshold
Parameters of Canny algorithm
Noise reduction– Size of Gaussian filter
Finding the intensity gradient of the image Non-maximum suppression Tracing edges through the image and hyste
resis thresholding– High threshold– Low threshold
OpenCV Implementation The Canny Operator
void cvCanny( const CvArr* img, CvArr* edges, double lowThresh, double highThresh, int apertureSize = 3);
void cvCanny( const CvArr* img, CvArr* edges, double lowThresh, double highThresh, int apertureSize = 3);
Example: Canny Edge Detector
DownloadTest Program
DownloadTest Program
Review:Derivatives of Image in 1D
( )I x
2 ( ) ( )I x I x
( )I I x
Edges can be characterized as either:– local extrema of I(x)– zero-crossings of 2I(x)
1D image
gradient
Laplacian
Laplacian
2 22
2 2
( , ) ( , )( , ) ( , )
I x y I x yI x y f x y
x y
A scalar isotropic.
Edge detection: Find all points for which
2I(x, y) = 0
No thinning is necessary.
Tends to produce closed edge contours.
Laplacian
2 22
2 2
( , ) ( , )( , )
I x y I x yI x y
x y
Discrete Laplacian Operators
2 22
2 2
( , ) ( , )( , )
I x y I x yI x y
x y
010
141
010
111
181
111
121
242
121
OpenCV Implementation The Discrete Laplacian Operators
2 22
2 2
( , ) ( , )( , )
I x y I x yI x y
x y
void cvLaplace( const CvArr* src, CvArr* dst, int apertureSize = 3);
void cvLaplace( const CvArr* src, CvArr* dst, int apertureSize = 3);
Example
2 22
2 2
( , ) ( , )( , )
I x y I x yI x y
x y
2 ( , )I x y( , )I x y
Laplician for Edge Detection
2 22
2 2
( , ) ( , )( , )
I x y I x yI x y
x y
2 ( , )I x y
Find zero-crossing on the Laplacian image.
Zero Crossing Detection
2 22
2 2
( , ) ( , )( , )
I x y I x yI x y
x y
There is a little bug in the above algorithm.
Try to design your own zero-crossing detection algorithm.
Example:Laplician for Edge Detection
DownloadTest Program
DownloadTest Program
2 22
2 2
( , ) ( , )( , )
I x y I x yI x y
x y
Laplacian for Image Sharpening
2 22
2 2
( , ) ( , )( , )
I x y I x yI x y
x y
2 ( , )I x y( , )I x y
*w
Example:Laplacian for Image Sharpening
2 22
2 2
( , ) ( , )( , )
I x y I x yI x y
x y
( , )I x y Sharpened Image
Laplacian of Gaussian (LoG)
( , )g x y 2I 2*I g
2 2
2 2
1( , ) exp
2 2
x yg x y
Gaussian:
2( , ) ( , )h x y g x y
2 2 2 22
4 2 2
1( , ) ( , ) 1 exp
2 2
x y x yh x y g x y
Some LoG Convolution Kernels
2 2 2 22
4 2 2
1( , ) ( , ) 1 exp
2 2
x y x yh x y g x y
0 0 0 0 1 1 1 0 0 0 0
0 0 1 2 3 3 3 2 1 0 0
0 1 2 4 5 5 5 4 2 1 0
0 2 4 5 1 2 1 5 4 2 0
1 3 5 1 14 24 14 1 5 3 1
1 3 5 2 24 40 24 2 5 3 1
1 3 5 1 14 24 14 1 5 3 1
0 2 4 5 1 2 1 5 4 2 0
0 1 2 4 5 5 5 4 2 1 0
0 0 1 2 3 3 3 2 1 0 0
0 0 0 0 1 1 1 0 0 0 0
0 1 1 2 2 2 1 2 0
1 2 4 5 5 5 4 2 1
1 4 5 3 0 3 5 4 1
2 5 3 12 24 12 3 5 2
2 5 0 24 40 24 0 5 2
2 5 3 21 24 12 3 5 2
1 4 5 3 0 3 5 4 1
1 2 4 5 5 5 4 2 1
0 1 1 2 2 2 1 2 0
0 0 1 0 0
0 1 2 1 0
1 2 16 2 1
0 1 2 1 0
0 0 1 0 0
Example:LoG for Edge Detection
by LoG
by Laplacian
Image Transforms
Hough Transforms
Goal of Hough Transforms
A technique to isolate the curves of a given shape / shapes in a given image
Classical Hough Transform – can locate regular curves like straight lines,
circles, parabolas, ellipses, etc.
Generalized Hough Transform – can be used where a simple analytic
description of feature is not possible
HT for Line Detection
x
y
y m bx
m
b
(m, b)
A line in xy-plane is a point in mb-plane.
HT for Line Detection
x
y 1 1y m bx
m
b
( , )x y
(m1, b1)2 2y m bx
(m2, b2)
3 3y m bx
(m3, b3)
All lines passing through a point in xy-plane is a line in mb-plane.
A line in xy-plane is a point in mb-plane.
b my x
HT for Line Detection
x
y 1 1y m bx
m
b
( , )x y
(m1, b1)2 2y m bx
(m2, b2)
3 3y m bx
(m3, b3)
All lines passing through a point in xy-plane is a line in mb-plane.A line in xy-plane is a point in mb-plane.
b my x ( , )x y
Given a point in xy-plane, we draw a line in mb-plane.
b my x
HT for Line Detection
x
y
m
b
A line in xy-plane is a point in mb-plane.
A line in xy-plane is then transformed in to a set of lines in mb-plane, which intersect at a common point.
Given a point in xy-plane, we draw a line in mb-plane.
y m bx (m, b)
HT for Line Detection
x
y
m
b
A line in xy-plane is a point in mb-plane.
A line in xy-plane is then transformed in to a set of lines in mb-plane, which intersect at a common point.
Given a point in xy-plane, we draw a line in mb-plane.
y m bx (m, b)
How to implement?How to implement?
Is mb representation suitable?Is mb representation suitable?
HT Line Detection by -representation
x
y
( cos , sin )
( , )x y
cos sinx y (, )
A line in xy-plane is a point in -plane.
HT Line Detection by -representation
x
y
A line in xy-plane is a point in -plane.
( , )x y1
2
3 4
43
2
1
cos sinx y
All lines passing through a point in xy-plane is a curve in -plane.
HT Line Detection by -representation
x
y
A line in xy-plane is a point in -plane.
( , )x y1
2
3 4
43
2
1
cos sinx y
All lines passing through a point in xy-plane is a curve in -plane.
Given a point in xy-plane, we draw a curve in -plane.
( , )x y
cos sinx y
HT Line Detection by -representation
x
y
A line in xy-plane is a point in -plane.
Given a point in xy-plane, we draw a curve in -plane.
A line in xy-plane is then transformed in to a set of curves in -plane, which intersect at a common point.
(, )
HT Line Detection by -representation
A line in xy-plane is a point in -plane.
Given a point in xy-plane, we draw a curve in -plane.A line in xy-plane is then transformed in to a set of curves in -plane, which intersect at a common point.
OpenCV Implementation Hough Line Transform
CvSeq* cvHoughLines2( CvArr* image, void* line_storage, int method, double rho, double theta, int threshold, double param1 = 0, double param2 = 0);
CvSeq* cvHoughLines2( CvArr* image, void* line_storage, int method, double rho, double theta, int threshold, double param1 = 0, double param2 = 0);
Example:Hough Line Transform
DownloadTest Program
DownloadTest Program
Hough Circle Transform
2 2 2( ) ( )x ya b r Circle equation:
x
y
r
a
b
0 0( , )x y
image space
0 0( , )x y
parameter space
Hough Circle Transform
2 2 2( ) ( )x ya b r Circle equation:
x
y
r
a
b
0 0( , )x y
image space
0 0( , )x y
parameter space
Cost ineffective & time consuming
Hough Gradient Method
2 2 2( ) ( )x ya b r Circle equation:
x
y
image space
0 0( , )x y
( , )a b
Parametric form: 0
0
cos
sin
x
y
a r
b r
0
0
cos
sin
a r
b r
x
y
0 0tan tanxb ya
Hough Gradient Method
2 2 2( ) ( )x ya b r Circle equation:
x
y
image space
0 0( , )x y
( , )a b
Parametric form: 0
0
cos
sin
x
y
a r
b r
0
0
cos
sin
a r
b r
x
y
0 0tan tanxb ya
The value of can be obtained from the edge detection
process.
The value of can be obtained from the edge detection
process.
Hough Gradient Method
Quantize the parameter space for the parameters a and b.
Zero the accumulator array M(a, b). Compute the gradient magnitude G(x, y)
and angle (x, y). For each edge (x0, y0) point in G(x, y), incre
ment all points in the accumulator array M(a, b) along the line
Local maxima in the accumulator array correspond to centers of circles in the image.
Circle equation:
x
y
image space
0 0( , )x y
( , )a b
0 0tan tanxb ya
0 0tan tanxb ya
2 2 2( ) ( )x ya b r
OpenCV Implementation Hough Circle Transform
CvSeq* cvHoughCircles( CvArr* image, void* circle_storage, int method, double dp, double min_dist, double param1=100, double param2=100 int min_radius=0, int max_radius=0);
CvSeq* cvHoughCircles( CvArr* image, void* circle_storage, int method, double dp, double min_dist, double param1=100, double param2=100 int min_radius=0, int max_radius=0);
Example:Hough Circle Transform
DownloadTest Program
DownloadTest Program
Image Transforms
Geometric Transforms
Geometric Transforms Stretch, Shrink, Warp, and Rotate
Scaling , Rotation, Translation
0
0x
y
s
s
x x
y y
cos sin
sin cos
x x
y y
x x x
y y y
Scaling
Rotation
Translation
Scaling , Rotation + Translation
0
0x
y
s
s
x x
y y
cos sin
sin cos
x x
y y
x x x
y y y
Scaling
Rotation
Translation
+Translation
+Translation
x
y
x
y
Homogeneous Coordinate
xx
y
w
w
wy
1
xx
yy
Scaling , Rotation + Translation
0
0x
y
x x x
y y y
s
s
cos sin
sin cos
x x x
y y y
Scaling
Rotation
+Translation
+Translation
0
01
x
y
s
s
xx
yy
cos sin
sin cos1
xx
yy
2 3 matrix
2 3 matrix
Affine Transformation
An affine transformation is any transformation that can be expressed in the form of a matrix multiplication followed by a vector addition. – In OpenCV the standard style of representing such a tran
sformation is as a 2-by-3 matrix.
00 01 0
10 11 1
a a bx
a bya
x
y
00 01 0
10 11 1 1
a a b
a a b
x
y
2 3 matrix
Affine Transformation
00 01 0
10 11 1 1
xa a bx
ay
y a b
GetAffineTransform
00 01 0
10 11 1 1
xa a bx
ay
y a b
Get Affine Transform
00 01 0
10 11 1 1
xa a bx
ay
y a b
Get 2D Rotation Matrix
WarpAffine00 01 0
10 11 1 1
xa a bx
ay
y a b
GetQuadrangleSubPix
Example: Affine Transform
DownloadTest Program
DownloadTest Program
GetQuadrangleSubPix
Sparse Affine Transformation
Perspective Transform
Perspective Transform
Perspective Transform
Affine Transform vs. Perspective Transform
00 01 0
10 11 1 1
xa a bx
ay
y a b
00 01 0
10 11 1
0 0 11 1
a a bx x
by a ya
Affine Transform:
x w
y w
w
Perspective Transform:
00 01 0
10 11 1
20 21 1 1
a a b
a a b
a
y
w a
x x
y
/
/
x x w
y y w
Get Perspective Transform
00 01 0
10 11 1
20 21 1 1
x w x x
y
a a b
a a yb
a a
w y
w w
WarpPerspective
00 01 0
10 11 1
20 21 1 1
x w x x
y
a a b
a a yb
a a
w y
w w
Sparse Perspective Transformation
00 01 0
10 11 1
20 21 1 1
x w x x
y
a a b
a a yb
a a
w y
w w
Image Transforms
Histogram Equalization
Graylevel Histogram of Image
Goal of Histogram Equalization
Goal of Histogram Equalization
Image Enhancement
Method Graylevel Remapping
0 1
fX(x)
x 0 1
fY(y)
y
y
xX Y
Probability Theory
y
xX Y
( )Xf xpdf
( )XF xcdf( )XY F X ~ (0,1)Y U
( )Xy F x
Example: Gaussian
Example: Gaussian
Demonstration
OpenCV Implementation
Example
DownloadTest Program
DownloadTest Program