7/30/2019 17 RANSAC

1/54

Feature Matching and RANSAC

15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005

with a lot of slides stolen from

Steve Seitz and Rick Szeliski

Krister Parmstrand

7/30/2019 17 RANSAC

2/54

Project #3 Showcase

mrom

7/30/2019 17 RANSAC

3/54

Project #3 Showcase

heegunl

7/30/2019 17 RANSAC

4/54

Project #3 Showcase

jmacalli

7/30/2019 17 RANSAC

5/54

Project #3 Showcase

lms

7/30/2019 17 RANSAC

6/54

Project #3 Showcase

cmcamero

7/30/2019 17 RANSAC

7/54

Project #3 Showcase

slim

7/30/2019 17 RANSAC

8/54

Project #3 Showcase

cmcamero

7/30/2019 17 RANSAC

9/54

Project #3 Showcase

nvyas

slim

7/30/2019 17 RANSAC

10/54

Project #3 Showcase

7/30/2019 17 RANSAC

11/54

Project #3 Showcase

mrom

7/30/2019 17 RANSAC

12/54

Project #3 Showcase

heegunl

7/30/2019 17 RANSAC

13/54

Project #3 Showcase

bhon

cmcamero

7/30/2019 17 RANSAC

14/54

Project #3 Showcase

chenyuwu

7/30/2019 17 RANSAC

15/54

Project #3 Showcase

slim

7/30/2019 17 RANSAC

16/54

Multi-Scale Oriented Patches

Interest points Multi-scale Harris corners

Orientation from blurred gradient

Geometrically invariant to rotation

Descriptor vector

Bias/gain normalized sampling of local patch (8x8)

Photometrically invariant to affine changes in intensity

[Brown, Szeliski, Winder, CVPR2005]

7/30/2019 17 RANSAC

17/54

Descriptor Vector

Orientation = blurred gradientRotation Invariant Frame

Scale-space position (x, y, s) + orientation ()

7/30/2019 17 RANSAC

18/54

Detections at multiple scales

7/30/2019 17 RANSAC

19/54

MOPS descriptor vector

8x8 oriented patch Sampled at 5 x scale

Bias/gain normalisation: I = (I )/

8 pixels40pixels

7/30/2019 17 RANSAC

20/54

Feature matching

?

F hi

7/30/2019 17 RANSAC

21/54

Feature matching

Exhaustive search for each feature in one image, look at all the other features in

the other image(s)

Hashing

compute a short descriptor from each feature vector, or hashlonger descriptors (randomly)

Nearest neighbor techniques

k-trees and their variants

Wh t b t tli ?

7/30/2019 17 RANSAC

22/54

What about outliers?

?

F t tli j ti

7/30/2019 17 RANSAC

23/54

Feature-space outlier rejection

Lets not match all features, but only these that havesimilar enoughmatches?

How can we do it?

SSD(patch1,patch2) < threshold How to set threshold?

F t tli j ti

7/30/2019 17 RANSAC

24/54

Feature-space outlier rejection

A better way [Lowe, 1999]: 1-NN: SSD of the closest match 2-NN: SSD of the second-closest match

Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN

That is, is our best match so much better than the rest?

F t tli j ti

7/30/2019 17 RANSAC

25/54

Feature-space outliner rejection

Can we now compute H from the blue points?

No! Still too many outliers

What can we do?

Matching features

7/30/2019 17 RANSAC

26/54

Matching features

What do we do about the bad matches?

RAndomSAmple Consensus

7/30/2019 17 RANSAC

27/54

RAndomSAmple Consensus

Select one match, count inliers

RAndomSAmple Consensus

7/30/2019 17 RANSAC

28/54

RAndomSAmple Consensus

Select one match, count inliers

Least squares fit

7/30/2019 17 RANSAC

29/54

Least squares fit

Find average translation vector

RANSAC for estimating homography

7/30/2019 17 RANSAC

30/54

RANSAC for estimating homography

RANSAC loop:

1. Select four feature pairs (at random)2. Compute homography H (exact)

3. Compute inliers where SSD(pi, Hpi) <

4. Keep largest set of inliers5. Re-compute least-squares H estimate on all of the

inliers

RANSAC

7/30/2019 17 RANSAC

31/54

RANSAC

7/30/2019 17 RANSAC

32/54

Example: Recognising Panoramas

M. Brown and D. Lowe,

University of British Columbia

Why Recognising Panoramas?

7/30/2019 17 RANSAC

33/54

Why Recognising Panoramas ?

Why Recognising Panoramas?

7/30/2019 17 RANSAC

34/54

Why Recognising Panoramas ?

1D Rotations ()

Ordering matching images

Why Recognising Panoramas?

7/30/2019 17 RANSAC

35/54

Why Recognising Panoramas ?

1D Rotations ()

Ordering matching images

Why Recognising Panoramas?

7/30/2019 17 RANSAC

36/54

Why Recognising Panoramas ?

1D Rotations ()

Ordering matching images

Why Recognising Panoramas?

7/30/2019 17 RANSAC

37/54

Why Recognising Panoramas ?

2D Rotations (, ) Ordering matching images

1D Rotations ()

Ordering matching images

Why Recognising Panoramas?

7/30/2019 17 RANSAC

38/54

Why Recognising Panoramas ?

1D Rotations ()

Ordering matching images

2D Rotations (, ) Ordering matching images

Why Recognising Panoramas?

7/30/2019 17 RANSAC

39/54

Why Recognising Panoramas ?

1D Rotations ()

Ordering matching images

2D Rotations (, ) Ordering matching images

Why Recognising Panoramas?

7/30/2019 17 RANSAC

40/54

y g g

Overview

7/30/2019 17 RANSAC

41/54

Feature Matching

Image Matching

Bundle Adjustment

Multi-band BlendingResults

Conclusions

RANSAC for Homography

7/30/2019 17 RANSAC

42/54

g p y

RANSAC for Homography

7/30/2019 17 RANSAC

43/54

g p y

RANSAC for Homography

7/30/2019 17 RANSAC

44/54

g p y

Probabilistic model for verification

7/30/2019 17 RANSAC

45/54

Finding the panoramas

7/30/2019 17 RANSAC

46/54

Finding the panoramas

7/30/2019 17 RANSAC

47/54

Finding the panoramas

7/30/2019 17 RANSAC

48/54

Finding the panoramas

7/30/2019 17 RANSAC

49/54

Homography for Rotation

7/30/2019 17 RANSAC

50/54

Parameterise each camera by rotation and focal length

This gives pairwise homographies

Bundle Adjustment

7/30/2019 17 RANSAC

51/54

New images initialised with rotation, focal length of bestmatching image

Bundle Adjustment

7/30/2019 17 RANSAC

52/54

New images initialised with rotation, focal length of bestmatching image

Multi-band Blending

7/30/2019 17 RANSAC

53/54

Burt & Adelson 1983

Blend frequency bands over range

Results

7/30/2019 17 RANSAC

54/54