depth estimate and focus recovery

40
Depth Estimate and Focus Recovery Presenter : Wen-Chih Hong Adviser: Jian-Jiun Ding Digital Image and Signal Processing Laboratory Graduate Institute of Communication Engineering National Taiwan University, Taipei, Taiwan, ROC 台台台台台 台台台台台台台台台台台台 111/05/10 1 DISP Lab @ MD531

Upload: prisca

Post on 11-Feb-2016

38 views

Category:

Documents


0 download

DESCRIPTION

Depth Estimate and Focus Recovery. Presenter : Wen-Chih Hong Adviser: Jian-Jiun Ding Digital Image and Signal Processing Laboratory Graduate Institute of Communication Engineering National Taiwan University, Taipei, Taiwan, ROC 台大電信所 數位影像與訊號處理實驗室. Outlines. Introduction - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Depth Estimate and Focus Recovery

Depth Estimate and Focus Recovery

Presenter : Wen-Chih Hong Adviser: Jian-Jiun Ding

Digital Image and Signal Processing LaboratoryGraduate Institute of Communication EngineeringNational Taiwan University, Taipei, Taiwan, ROC台大電信所 數位影像與訊號處理實驗室

112/04/22 1DISP Lab @ MD531

Page 2: Depth Estimate and Focus Recovery

Outlines Introduction Binocular version systems

Stereo Monocular version systems

DFF DFD

Other method Conclusions References

112/04/22 2DISP Lab @ MD531

Page 3: Depth Estimate and Focus Recovery

Introduction Depth is an important information for

robot and the 3D reconstruction. Image depth recovery is a long-term

subject for other applications such as robot vision and the restorations.

Most of depth recovery methods based on simply camera focus and defocus.

Focus recovery can help users to understand more details for the original defocus images.

112/04/22 3DISP Lab @ MD531

Page 4: Depth Estimate and Focus Recovery

Introduction Categories of depth estimation

Monocular

Depth from defocus (DFD)

Depth from focus (DFF)

Binocular Stereo focus

112/04/22 4DISP Lab @ MD531

Page 5: Depth Estimate and Focus Recovery

Introduction Categories of depth estimation

Active : Sending a controlled energy beam Detection of reflected energy

Passive: Image-based

112/04/22 5DISP Lab @ MD531

Page 6: Depth Estimate and Focus Recovery

Introduction Geometric on imaging

1 1 1

lD F

F

D/2

F

us

2R : R>0

sensor

vBiconvex

112/04/22 6DISP Lab @ MD531

Page 7: Depth Estimate and Focus Recovery

Binocular version systems The flow chart to binocular depth

estimation. Depth map HVS modeling Edge detection Correspondence Vengeance control Gaze control Depth map

112/04/22 7DISP Lab @ MD531

Page 8: Depth Estimate and Focus Recovery

Binocular version systems Vengeance movement :

is some kind of slow eye movement that two eyes move in different directions.

But corresponding problem Gazing point(Corresponding point)

Baseline (B)

B/2B/2

Depth (u)

2 2 22 2

2 2

sin cos cos4sin sin

L R L R

L R L R

u B

112/04/22 8DISP Lab @ MD531

Page 9: Depth Estimate and Focus Recovery

Binocular version systems Complex model

RdiffLdiff

Corresponding point

Depth (u)

Right vision

Left vision Baseline (B)

(xR, yR)(xL, yL)

Figure 3.3 A more complete triangulation geometry for the binocular vision.We have to realize how much departure between the optical axis and the direction

of the

tan tan

tan tanLdiff Rdiff

Ldiff Rdiff

Bu

1tan tan / 2Rdiff Rx W

1tan tan / 2Ldiff Lx W

112/04/22 9DISP Lab @ MD531

Page 10: Depth Estimate and Focus Recovery

Binocular version systems Corresponding problem But more accuracy

112/04/22 10DISP Lab @ MD531

Page 11: Depth Estimate and Focus Recovery

Monocular version systems Depth from focus

Depth form defocus

112/04/22 11DISP Lab @ MD531

Page 12: Depth Estimate and Focus Recovery

Depth from Focus Taking pictures at different observer distance or object

distance We need an estimator to measure degree on focus

Using Laplacian operator

Such operator point to a measurement on a single pixel influence, a sum of Laplacian operator is needed:

( , ) ( , , )* ( , )kg x y h x y t f x y

2 22

2 2( , ) ( , )

( , ) k kk

g x y g x yg x y

x y

, ,j k i k

v j k u i k

n i j ML u v

112/04/22 12DISP Lab @ MD531

Page 13: Depth Estimate and Focus Recovery

Depth from Focus Gaussian interpolation

Figure 4.4 Gaussian interpolation to a measure curve, Nk≧ Nk-1, Nk≧ Nk+1

displacementdk

[SML]

NPFocus measure

Nk

Nk-1

dk-1dp

Measured curveIdeal condition

dk+1

Nk+1

112/04/22 13DISP Lab @ MD531

Page 14: Depth Estimate and Focus Recovery

Depth from Focus Range from focus

using

Take pictures along the axis Find the image having highest frequency Need more than 10 images (monocular)

1 1 1f D v

112/04/22 14DISP Lab @ MD531

Page 15: Depth Estimate and Focus Recovery

Depth from Focus We use Gaussian interpolation to form

a set of approximations The depth solution dp from above

Gaussian:

2 21 1

1 1 1

2 21 1

1 1 1

ln ln

2 ln ln ln ln

ln ln

2 ln ln ln ln

k k k kp

k k k k k k

k k k k

k k k k k k

N N d dd

d d N N N N

N N d d

d d N N N N

112/04/22 15DISP Lab @ MD531

Page 16: Depth Estimate and Focus Recovery

Depth from Defocus Due to geometric optics, the intensity inside the blur

circle should be constant. Considering of aberration and diffraction and so on, we

easily assume a blurring function: : diffusion parameter

Diffusion parameter is related to blur radius: derived from triangularity in geometric optics

For easy computation, we assume that foreground has equal-diffusion, background has equal-diffusion and so on

However, this equal-focal assumption will be a problem

2 2

2 2

1, exp2 2

x yh x y

112/04/22 16DISP Lab @ MD531

Page 17: Depth Estimate and Focus Recovery

Depth from Defocus Blurring model

Blurring radius

112/04/22 17DISP Lab @ MD531

Page 18: Depth Estimate and Focus Recovery

Depth from Defocus Blurring model

1 0

0

br rv v v

2 0

0

br rv v v

0v v v

112/04/22 18DISP Lab @ MD531

Page 19: Depth Estimate and Focus Recovery

Depth from Defocus Blurring model

1 0

0

br rv v v

2 0

0

br rv v v

0v v v

112/04/22 19DISP Lab @ MD531

Page 20: Depth Estimate and Focus Recovery

Depth from Defocus Blurring model

1 0

0

br rv v v

2 0

0

br rv v v

0v v v

112/04/22 20DISP Lab @ MD531

Page 21: Depth Estimate and Focus Recovery

Depth from Defocus Blurring model

when

blur radius is independent of the location of the point source on the object plane at depth

01 2 0 ( 1)b b

vr r r rv

0

01 2 0 ( 1)b b

vr r r rv

112/04/22 21DISP Lab @ MD531

Page 22: Depth Estimate and Focus Recovery

Depth from Defocus Blurring model

Using and

We get So diffusion parameter:

1 1 1

lv F D 0

1 2 0 ( 1)b bv

r r r rv

0 00

1 1 1( )bl

r r vF v D

1 1 1( )m m mlm m

r vF v D

112/04/22 22DISP Lab @ MD531

Page 23: Depth Estimate and Focus Recovery

Depth from Defocus Depth recovery

Eliminating D from m=1,2 we get

where and

1 2

1 1 1( )m m mlm m

r vF v D

1 1

2 2

rvr v

1 11 1 2 2

1 1 1 1( )l l

rvF v F v

112/04/22 23DISP Lab @ MD531

Page 24: Depth Estimate and Focus Recovery

Depth from Defocus Depth recovery

From Take F.T.:

The F.T. of Gaussian is Gaussian

( , ) ( , ) ( , )k kg x y h x y f x y

( , ) ( , ) ( , )k kG w v H w v F w v

2 2

2 2

1, exp2 2

x yh x y

2 2 2 211 2

2

( , ) 1exp[ ( )( )]( , ) 2

G w v w vG w v

112/04/22 24DISP Lab @ MD531

Page 25: Depth Estimate and Focus Recovery

Depth from Defocus Depth recovery

Take the log

Using the relationship between them

we get

1 2 2 2 2

2 2( 1) 2 C

2 2 11 2 2 2

2

( , )2 log( , )

G v Cv G v

2 2 2 211 2

2

( , ) 1exp[ ( )( )]( , ) 2

G w v w vG w v

112/04/22 25DISP Lab @ MD531

Page 26: Depth Estimate and Focus Recovery

Depth from Defocus Depth recovery

let apha=1 we obtain the value of sigma-2

Find out the depth D

2 2 22 2( 1) 2 C

1 1

2 2

1rvr v

1 1 1( )m m mlm m

r vF v D

112/04/22 26DISP Lab @ MD531

Page 27: Depth Estimate and Focus Recovery

Depth from Defocus The main sources of range errors in

DFD

Inaccurate modeling of the optical system. Windowing for local feature analysis. Low spectral content in the scene being

images. Improper calibration of camera parameters. Presence of sensor noise.

112/04/22 27DISP Lab @ MD531

Page 28: Depth Estimate and Focus Recovery

Depth from Defocus Block shift-variant blur model

Consider the interaction of sub-images Define the neighborhood function

Indeed, the image we observed is

compared with

{ , 1,..., ,..., 1, }in i J i J i i J i J

( ) ( )ni

i

i h in

g m f m a d

( , ) ( , ) ( , )k kg x y h x y f x y

112/04/22 28DISP Lab @ MD531

Page 29: Depth Estimate and Focus Recovery

Depth from Defocus Space-variant filtering models for

recovering depth Using complex spectrogram and P.W.D. Complex Spectrogram:

2 1( ) '( , ') ( ') 'x t h t t x t dt

*''( , ') ( ' ) '( , ) exp( ( '))2kh t t u t t H t j t t d

112/04/22 29DISP Lab @ MD531

Page 30: Depth Estimate and Focus Recovery

Depth from Defocus Space-variant filtering models for

recovering depth C.S.:

g_1/g_2

where

( , ) ( , ) ( , )ig f iC t C t H t

2 1( , ) ( , ) ( , )g gC t C t H t

2

1

( , )( , )( , )

H tH tH t

112/04/22 30DISP Lab @ MD531

Page 31: Depth Estimate and Focus Recovery

Depth from Defocus Space-variant filtering models for

recovering depth objective function:

Drawback: No consider the intersection of pixels there will

be interrupt in border. Regularized solution.

2 1

' 1 2 2 2 2

( ) 0

min ( ( ) ( ) exp( ( ) ( )))N

g i g is i k

C k C k k s i

2 22 1( ) ( ) ( )s i i i

112/04/22 31DISP Lab @ MD531

Page 32: Depth Estimate and Focus Recovery

Depth from Defocus No corresponding problem Less accuracy

S.V. > B.S.V. Blocking Trade-off

Blocking size Too large: less accuracy Too small: noise

112/04/22 32DISP Lab @ MD531

Page 33: Depth Estimate and Focus Recovery

Other method Structure from motion Shape from shading

ML Estimation of Depth and Optimal camera settings Recursive computation of depth from multiple images

112/04/22 33DISP Lab @ MD531

Page 34: Depth Estimate and Focus Recovery

Other method Structure from motion

Using the relative motion between object and camera to find out surface information

Corresponding problem (binocular) Find out what motion of camera

112/04/22 34DISP Lab @ MD531

Page 35: Depth Estimate and Focus Recovery

Other method Shape from shading

Need to know the reflectance Find the sliding rate and blindness

112/04/22 35DISP Lab @ MD531

Page 36: Depth Estimate and Focus Recovery

Focus recovery

SML measurement

Defocused image pair

Full focused image

Maximum value searching

Depth measurement of a point

Small aperture construction

Linear canonical transform based on constructed

optical system

focal point

Using the specific depth to retrieve imaging

distance

112/04/22 36DISP Lab @ MD531

Page 37: Depth Estimate and Focus Recovery

Conclusions Binocular stereo method

high accuracy Absolute depth information Complexity computation Corresponding problem

Structure form motion Nonlinear problem Corresponding problem

Shape from shading Very difficult method Active method

112/04/22 37DISP Lab @ MD531

Page 38: Depth Estimate and Focus Recovery

Conclusions Range from focus :

Slowly More than 10 images

depth from defocus : Easy method Less accuracy

112/04/22 38DISP Lab @ MD531

Page 39: Depth Estimate and Focus Recovery

References and future work1) Y. C. Lin, Depth Estimation and Focus Recovery, Master

thesis, National Taiwan Univ., Taipei, Taiwan, R.O.C, 20082) Subhasis Chaudhuri, A.N. Rajagopalan, ”Depth From Defocus:

A Real Aperture Imaging Approach. ” Springer-Verlag. New York, 1999.

3) M. Subbarao, “Parallel depth recovery by changing camera parameters,” Second International Conference on Computer Vision 1988, pp. 149-155, Dec. 1988.

4) M. Subbarao and T. C. Wei, “Depth from defocus and rapid autofocusing: a practical approach,” IEEE Conferences on Computer Vision and Pattern Recognition, pp. 773-776, Jun. 1992.

5) A. N. Rajagopalan and S. Chaudhuri, “A variational approach to recovering depth from defocused images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, pp. 1158-1164, Oct. 1997.

112/04/22 39DISP Lab @ MD531

Page 40: Depth Estimate and Focus Recovery

The end

112/04/22 DISP Lab @ MD531 40