Smoothly Varying Affine Stitching [CVPR 2011]

Ph.D. Student, Chang-Ryeol Lee

February 10, 2013

2

Introduction– Motivation– Problem

Related works– Dynamosaics: Video Mosaics with Non-Chronological Time [CVPR

2005]

Proposed method– Smoothly Varying Affine Stitching [CVPR 2011]

Expeirments

Contents

3

Motivation– Typical camera FOV: 50˚ X 35˚

Introduction

4

Motivation– Typical camera FOV: 50˚ X 35˚– Human FOV: 200˚ X 135˚

Introduction

5

Motivation– Typical camera FOV: 50˚ X 35˚– Human FOV: 200˚ X 135˚– Panoramic view: 360˚ X 180˚

Introduction

6

Impressive

Introduction

7

Problem– Usually generating using rotating the camera around the

center of projection:

Introduction

The mosaic has a natural interpretation in 3D The images are reprojected onto a common plane The mosaic is formed on this plane

8

Problem: Changing Camera Center

Introduction

synthetic PP

PP1

PP2

9

Problem: Changing Camera Center

Introduction

Pics from Internet

10

Dynamosaics: Video Mosaics with Non-Chronological Time [CVPR 2005]– Shmuel Peleg (Hebrew University, Israel)

Motivation– Satellites create panoramas by scanning

• 1D sensor• Rotation & Translation• How can this idea be utilized?

Related works

11

Push broom stitching

Related works

t t+1 t+2

12

Time-Space Cube– Align the images– Create Push-broom mosaics by combining the image pieces– Different Cuts can create different mosaics

Related works

1 2 3 4 5

13

Push broom distortion – x-axis: Orthographic Projection– y-axis: Perspective Projection– y shrinks as Z increases, x doesn’t

Experimental result

Related works

x xy y

fz z

14

Smoothly Varying Affine Stitching [CVPR 2011]– Loong-Fah Cheong (NUS)

Work Assumption– Most scenes can be modeled as having smoothly varying depth– A global affine has general shape preservation

Proposed method

15

Proposed method System overview

16

Proposed method The affine stitching field

0 06 11 ; 2i i iv

a b b

17

Proposed method Algorithm to compute stitching field

– Input: 1. M Base image features

2. N Target image features

3. Global affine matrix

– Output:1. Converged affine matrix

0ib

0 jt

convergedA

globalA T

0 0, , , SIFT descriptorsi j x yb t

18

Proposed method Algorithm to compute stitching field

– Cost function

– Notation• Affine parameters

• Stitched feature points by

01: 1:log |N ME P A t b Ψ A

(1) (2) (3)2

(4) (5) 0 (6)

12

i i iS

i ii i i

SS S S

a a a0

a ab b a

00 I

6 1,...,T

M M A a a i global i a a a 0 06 11 ; 2i i iv

a b b

ia

19

Proposed method Algorithm to compute stitching field

– Cost function

– Notation• Robust Gaussian mixture

• Smoothness regularization

201: 1: 0 1:

11

| , 2N M

N M j M t tij

P g

t b t b

2

2

'

'min

'v

vd

g

RΨ A

2

22( , )z

g e

z

'

'

v

g

01: 1:log |N ME P A t b Ψ A

: Fourier transform of v

: Fourier transform of Gaussian

20

Proposed method Algorithm to compute stitching field

– Cost function

– Minimization by EM style optimization

– Estimated stitching field map

01: 1:log |N ME P A t b Ψ A

T2 1 0 (1) 0 (2)

1

( ) ( [ ] )M

i i ii

v a g

z z b b

21

Re-shoot

Applications

22

Re-shoot

Applications

23

Experiments

Panoramic stitching

24

Experiments

Matching

25

Thank you!

* This material is based on Raz Nossek‘s Image Registration & Mosaicing.