Computer Vision

More Image Features

2015. 4. 12Hyunki Hong

School of Integrative Engi-neering

Contents

• Line & Curve Detection

• Hough Transform

• Fitting Ellipses to Image Data

S. Seitz 자료 등 참조

Voting schemes

• Let each feature vote for all the models that are compatible with it

• Hopefully the noise features will not vote consistently for any single model

• Missing data doesn’t matter as long as there are enough fea-tures remaining to agree on a good model

Hough transform

• an early type of voting scheme• general outline:

– Discretize parameter space into bins.– For each feature point in the image, put a vote in every bin in the pa-

rameter space that could have generated this point.– Find bins that have the most votes.

P. V. C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and In-strumentation, 1959

Image space Hough parameter space

Parameter space representation

• A line in the image corresponds to a point in Hough space

Image space Hough parameter space

Parameter space representation

• What does a point (x0, y0) in the image space map to in the Hough space?

Image space Hough parameter space

Parameter space representation

• What does a point (x0, y0) in the image space map to in the Hough space?– Answer: the solutions of b = –x0m + y0

– This is a line in Hough space.

Image space Hough parameter space

점 지나는 선분 → 점

Parameter space representation

• Where is the line that contains both (x0, y0) and (x1, y1)?

Image space Hough parameter space

(x0, y0)

(x1, y1)

b = –x1m + y1

Parameter space representation

• Where is the line that contains both (x0, y0) and (x1, y1)?

– It is the intersection of the lines b = –x0m + y0 and b = –x1m + y1

Image space Hough parameter space

(x0, y0)

(x1, y1)

b = –x1m + y1

b = –x0m + y0

• Problems with the (m, b) space:– Unbounded parameter domain– Vertical lines require infinite m

Parameter space representation

tan-1 90° = ∞

• Problems with the (m,b) space:– Unbounded parameter domain– Vertical lines require infinite m.

• Alternative: polar representation

Parameter space representation

sin cos yx

Each point will add a sinusoid in the (, ) parameter space.

Algorithm outline

• Initialize accumulator H to all zeros.• For each edge point (x, y) in the image

For θ = 0 to 180 ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1

endend

• Find the value(s) of (θ, ρ) where H(θ, ρ) is a local maxi-mum.– The detected line in the image is given by

ρ = x cos θ + y sin θ

ρ

θ

features votes

Basic illustration

http://en.wikipedia.org/wiki/Hough_transform

Square Circle

Other shapes

Several lines

A more complicated image

http://ostatic.com/files/images/ss_hough.jpg

features votes

Effect of noise

• Peak gets fuzzy and hard to locate

Effect of noise

• Number of votes for a line of 20 points with increasing noise:

Random points

• Uniform noise can lead to spurious peaks in the array

features votes

Random points

• As the level of uniform noise increases, the maximum number of votes increases too:

Dealing with noise

• Choose a good grid / discretization

– Too coarse: large votes obtained when too many different lines

correspond to a single bucket

– Too fine: miss lines because some points that are not exactly

collinear cast votes for different buckets

• Increment neighboring bins (smoothing in accumulator array)• Try to get rid of irrelevant features

– Take only edge points with significant gradient magnitude

Incorporating image gradients

• Recall: when we detect an edge point, we also know its gradient direction

• But this means that the line is uniquely determined!• Modified Hough transform:• Algorithm For each edge point (x, y)

θ = gradient orientation at (x, y)ρ = x cos θ + y sin θH(θ, ρ) = H(θ, ρ) + 1

end

Hough transform for circles

• How many dimensions will the parameter space have?• Given an oriented edge point, what are all possible bins that it

can vote for?• In a two dimensional space, a circle can by described by:

- If a 2D point (x, y) is fixed, then the parameters can be found. - The parameter space would be three dimensional, (a, b, r). - In the 3D space, the circle parameters can be identified by the in-

tersection of many conic surfaces that are defined by points on the 2D circle. 1. The first stage: fixing radius then find the optimal center of circles in

a 2D parameter space. 2. The second stage: to find the optimal radius in a one dimensional pa-

rameter space.

(a, b) : the center of the circle r : the radius

Consider 4 points on a circle in the original image and the circle Hough transform. Note that the radius is assumed to be known. For each (x, y) of the four points (white points) in the orig-inal image, it can define a circle in the Hough parameter space centered at (x, y) with radius r

Find circle parameter with unknown radius: Since the parameter space is 3D, the accumulator matrix would be 3D, too. We can iterate through possible radiuses and for each radius. Finally, find the local maxima in the 3D accumulator matrix.

Line & Curve Detection

• Line detection can be performed directly: template matching : look for the peaks of the convolution between the image and a

set of masks matched to long, linear edges in all possible ori-entations

• Line and curve detection splits logically into two sub-prob-

lems.

- Grouping: Which image points compose each instance of the target

curve in the images?

- Model fitting: Given a set of image points probably belonging to a

single instance of the target curve, find the best curve interpolating

the points.

•  detect complex patterns of points in binary images

• mapping a difficult pattern detection problem into a simple peak de-

tection problem in the space of the parameters of the curve

• two basic steps of HT for lines 1. Transform line detection into a line intersection problem

Any line y = mx + n is identified by a unique parameter pair (m, n).

→ The line is represented by a point in the m, n plane (the parameter

space).

Conversely, any point p = [x, y]T in the image corresponds to a line n = x(-

m) + y in parameter space, which m & n vary, represents all possible image

lines through p.

→ A line defined by N collinear image points, p1, .., pN, is identified in pa-

rameter space by the intersection of the lines associated with p1, .., pN

Hough Transform (HT)

ex. Lines and simple curves

• The original Hough transform : to detect lines & curves

(a) Image space (b) k, q parameter space

• Any straight line in the image

→ a single point in the k, q parameter space

• Accumulator array & threshold

2. Transform line intersection in a simple peak detection problem, or

search for a maximum. a. Divide the m, n plane into a finite grid of cells, the resolution of which de-

pends on the accuracy we need.

b. For each image pixel pi, increment all counters on the corresponding line in

parameter space.

c. The image line is identified by the peak of c(m, n) in parameter space.

• The polar representation solves two problems. 1. Both m and n can take on values in [-∞, ∞], which implies that we can-

not sample the whole paramter space.

→ a limit to the size of the discrete paramter space

2. (m, n) parameterization does not capture the bundle x = k.

Hough Transform (HT)

sincos yx

• Effect of nonlinear contours: curved contours, or just noise in-

troduced by the edge detector threshold c(m, n)

• Because of pixelization and the limited accuracy of edge detec-

tion, not all the parameter-space lines corresponding to the

points of an image line intersect at the same point.

Image(64×64) containing two lines, sampled irregularly, and several random points (left)Plot of the contours in the corresponding parameter space (how many points contribute to each cell(m, n) (right)

HT Algorithm

• The HT generalized to detect curves

1. curves y = f(x, a), where a = [a1, …, ap]T is a vector of p parameters.

• HT

1. a voting algorithm

a. Each point “votes” for all combinations of parameters.

b. The array of counters in parameter space can be regarded as a histogram.

c. The final total of votes indicates the relative likelihood of the hypothsis.

“A curve with paramter set m existis in the image.”

2. pattern (= the class of curves) matching

3. merits: 1) it copes well with occlusion. 2) it’s relatively robust to noise.

• Major limitations

1. the rapid increase of the search time with the number of parameters in the

curve’s representation

2. Non-target shapes can produce spurious peaks in parameter space.

N

ii

aD

1

2),(min ap

• The ellipse detectors take an image of edge points in input, and find the best ellipse fitting the points.

• Ellipse fitting identifies a set of image points plausibly belonging to a single arc of ellipse.

• Algorithm 1. Let p1, …, pN be a set of N image points, pi= [xi, yi]

T. Let x = [x2, xy, y2,

x, y, 1]T, p = [x, y]T, and f(p, a) = xT a = ax2 + bxy + cy2 + dx + ey + f =

0, the implicit equation of the generic ellipse, characterized by the pa-

rameter vector a = [a, b, c, d, e, f]T

2. Find the parameter vector, a, associated to the ellipse which fits p1, …,

pN best in the least squares sense, as the solution of ,

where D(pi, a) is a suitable distance.

What is a suitable distance? ( Euclidean distance & algebraic distance)

Fitting Ellipses to Image Data

f(p, a) : generic conic

• A conic section (or just conic) is a curve obtained as the intersec-tion of a cone with a plane.

• In the Cartesian coordinate system, the graph of a quadratic equa-tion in two variables is always a conic section:

• The quadratic can be written as

Conic Section

Parabola, Circle, ellipse, and Hyperbola

• Minimize the Euclidean distance between the ellipse & the mea-sured points.

, under the constraint that p belongs to the ellipse: • Let us try Lagrange multipliers to solve (1). • An objective function : and set which yield (2)

• Since we do not know p, we try to express it as a function of com-putable quantities. Use two approximations

1. consider a first-order approximation of the curve (3) 2. assume that the pi are close enough to the curve, so that

N

ii

a1

2min pp

),(21

2appp fL

N

ii

0y

L

x

L

),(][),(),(0 apppapap iT

ii fff

),( appp ii f

0),( apf

Euclidean Distance Fit

(1)

)()( iff pp

Go to 37 page

Linear Approximation

• an approximation of a general function using a linear function (more precisely, an affine function)

- They are widely used in the method of finite differences to produce first order methods for solving or approximating solutions to equa-tions.

- Given a twice continuously differentiable function f of one real variable, Taylor's theorem for the case n = 1 states that

, where R2 is the remainder term.

- The linear approximation is obtained by dropping the remainder:

- This is a good approximation for x when it is close enough to a; since a curve, when closely observed, will begin to resemble a straight line.

Tangent line at (a, f(a))

예 ) y = x2

• rewrite Eq. (2) as

which, plugged into Eq. (3), gives

• Substituting in Eq. (2), we finally find

),( appp ii f

2),(

),(

ap

ap

i

i

f

f

),(

),(

ap

appp

i

ii f

f

cf. Another method: Algebraic distance turns into a linear problem with no further approximations

N

ii

aD

1

2),(min ap

• In mathematical optimization, Lagrange multiplier method is a strategy for finding the local maxima and minima of a function subject to equality constraints.

• For instance, consider the optimization problem maximize f(x, y) subject to g(x, y) = c.

Lagrange multiplier

Find x and y to maximize f(x, y) subject to a con-straint (shown in red) g(x, y) = c.

Contour map. The red line shows the constraint g(x, y) = c. The blue lines are contours of f(x, y). The point where the red line tangentially touches a blue contour is our solution. Since d1 > d2, the so-lution is a maximization of f(x, y).

when one wishes to maximize or minimize a function subject to fixed outside conditions or constraints.

• We need both f and g to have continuous first partial derivatives.

• Introduce a new variable (λ) called a Lagrange multiplier and study

the Lagrange function (or Lagrangian) defined by

where λ term may be either added or subtracted. If f(x0, y0) is a max-

imum of f(x, y) for the original constrained problem, then there ex-ists λ0 such that (x0, y0, λ0) is a stationary point for the Lagrange func-

tion (stationary points: those points where the partial derivatives of

Λ are zero, i.e. Λ = 0). ∇

Lagrange multiplier

A stationary point or critical point is a point of the do-main of a differentiable function, where the derivative is zero.

• consider the 2D problem: maximize f(x, y) subject to g(x, y) = c.• Lagrange multipliers relies on the intuition that at a maximum f(x, y)

cannot be increasing in the direction of any neighboring point where g = c.

• Suppose we walk along the contour line with g = c. We are interested in finding points where f does not change as we walk, since these points might be a maximum.

→ we could be following a contour line of f, since by definition f does not change as we walk along its contour lines. This would mean that the contour lines of f and g are parallel here

: for some λ where : the respective gradients

• auxiliary function and solve

Lagrange multiplier

• Example 1: Suppose we wish to maximize f(x, y) = x + y sub-ject to the constraint x2 + y2 =1. The feasible set is the unit cir-cle, and the level sets of f are diagonal lines (with slope -1), so we can see graphically that the maximum occurs at , and that the minimum occurs at .

1. Using the method of Lagrange multipliers,

g(x, y) – c = x2 + y2 –1.

2. Setting the gradient yields the system of equations:

3. Evaluating the objective function f at these points yields the maximum & the minimum:

2

2,

2

2

2

2,

2

2

대입하면 ,

2

2,

2

2 stationary points: and

2

2,

2

2

2,2

Generalized Hough transform

• We want to find a shape defined by its boundary points and a reference point.

D. Ballard, Generalizing the Hough Transform to Detect Arbitrary Shapes, Pattern Recognition 13(2), 1981, pp. 111-122.

a

p

Generalized Hough transform

• We want to find a shape defined by its boundary points and a reference point.

• For every boundary point p, we can compute the displacement vector r = a – p as a function of gradient orientation θ

D. Ballard, Generalizing the Hough Transform to Detect Arbitrary Shapes, Pattern Recognition 13(2), 1981, pp. 111-122.

a

θ r(θ)

Generalized Hough transform

• For model shape: construct a table indexed by θ storing dis-placement vectors r as function of gradient direction

• Detection: For each edge point p with gradient orientation θ:– Retrieve all r indexed with θ– For each r(θ), put a vote in the Hough space at p + r(θ)

• Peak in this Hough space is reference point with most sup-porting edges

• Assumption: translation is the only transformation here, i.e., orientation and scale are fixed