an interactive background blurring mechanism and its applications
DESCRIPTION
An Interactive Background Blurring Mechanism and Its Applications. NTU CSIE Yan Chih -Yu Advisor: Wu Ja -Ling, Ph.D. Outline. Introduction Related Work Method Object Segmentation Depth Map Generation Image Defocus Experimental Result Applications Conclusion. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
An Interactive Background Blurring Mechanism and Its Applications
NTU CSIE
Yan Chih-Yu
Advisor: Wu Ja-Ling, Ph.D.
2
3
Outline IntroductionRelated WorkMethod
◦Object Segmentation◦Depth Map Generation◦Image Defocus
Experimental ResultApplicationsConclusion
Introduction
Real world
Photo plane
Depth of field◦Circle of confusion
5
Photo plane
Introduction
Real light
Circle of Confusion
Out of focus
Depth of field◦Circle of confusion
Introduction Depth of field
Real light
Circle of Confusion- Readability range
Focus plane Photo planeOut of focus – blur range
Depth of field
Introduction
Real world
Photo plane
Shallow focus
8
IntroductionShallow focus
◦Highlight the subject by softening background diffusion
Deep focus by DC
NIKON E4300 (2003)
Shallow focus by DSLR
NIKON D90(2008)
Method
Lazy SnappingLazy Snapping
Alpha MattingAlpha Matting
Face DetectionFace Detection
Perspective BoxPerspective Box
Pop-up CardPop-up Card
Camera SettingsCamera Settings
Defocus BlurDefocus Blur
Privacy PreservingPrivacy Preserving
Photo BrowserPhoto Browser
Object Segmentation
Depth Map Generation
Applications
Image Defocus
10
Method
Segment◦Lazy snapping◦Mean shift◦Alpha matting◦Face detection
Resize to 20%
Do hard graph cut by lazy snapping
User Stroke
Mean shift
Do hard graph cut by lazy snapping
User Stroke
Resize to 20%
Mean shift
Segmentation tri-Map
Segmentation alpha Map
Lazy Snapping. ACM Trans. On Graphics 2004.Mean Shift: A Robust Approach Toward Feature Space Analysis. PAMI 2002.A Bayesian Approach to Digital Matting. CVPR 2001.
11
MethodDepth map
◦ Perspective Box Vanish point Rear wall
Depth Map
Perspective Box
Normal vector
Pop-up card
Depth Map
Perspective Box
Normal vector
Pop-up card
Tour Into the Picture: Using a spidery mesh user interface to make animation from a single image. SIGGRAPH 1997
12
MethodDepth map
◦ Pop-up card
Depth Map
Perspective Box
Normal vector
Pop-up card
Depth Map
Normal vector
Pop-up card
13
MethodDepth map
◦ Normal vector
Depth Map
Perspective Box
Normal vector
Pop-up card
Depth Map
Normal vector
14
Method
Image defocus◦Blur circle diameter
Blur circle diameter
Segment Map
Depth Map
Camera setting
Aperture sizeFocal length of the
lensDistance of focus
u v
bd
vbub
u v
bd
vbub
(a) (b)𝑏= 𝑓𝑁ȁ�(𝑢𝑏 − 𝑢)𝑓ȁ�𝑢𝑏(𝑢− 𝑓) 1𝑢+ 1𝑣 = 1𝑓
15
MethodDefocus blur
◦bokeh
𝐼𝑑𝑒𝑓𝑜𝑐𝑢𝑠 ሺ𝑖,𝑗ሻ= σ𝐸𝑛ሺ𝑖,𝑗ሻσ𝑤𝑒𝑖𝑔ℎ𝑡𝑛(𝑖,𝑗) + 𝛼𝑛(𝑖,𝑗)
𝐸𝑛ሺ𝑖,𝑗ሻ= 𝐼ሺ𝑥𝑛,𝑦𝑛ሻ 𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅𝑒𝑔𝑖𝑜𝑛(𝑥𝑛,𝑦𝑛,𝑠,𝑏)Τ
𝑤𝑒𝑖𝑔ℎ𝑡𝑛ሺ𝑖,𝑗ሻ= 1 𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅𝑒𝑔𝑖𝑜𝑛(𝑥𝑛,𝑦𝑛,𝑠,𝑏)Τ
𝛼𝑛(𝑖,𝑗) =൜𝑤𝑒𝑖𝑔ℎ𝑡𝑛(𝑖,𝑗), 𝐼ሺ𝑥𝑛,𝑦𝑛ሻ< 𝑇ℎ𝑏𝑜𝑘𝑒ℎ0, 𝐼ሺ𝑥𝑛,𝑦𝑛ሻ≥ 𝑇ℎ𝑏𝑜𝑘𝑒ℎ
𝑤𝑒𝑖𝑔ℎ𝑡𝑛ሺ𝑖,𝑗ሻ= 1𝑎𝑟𝑒𝑎 𝑜𝑓 𝑅𝑒𝑔𝑖𝑜𝑛ሺ𝑥𝑛,𝑦𝑛,𝑠,𝑏ሻ× 𝑝𝑜𝑠𝑖𝑡𝑖𝑜𝑛(𝑥𝑛,𝑦𝑛)𝑙𝑢𝑚𝑖𝑛𝑜𝑢𝑠(𝑥𝑛,𝑦𝑛)
……
……E1(i,j)
I(x1,y1) I(x2,y2) I(x3,y3)
I(xn,yn) I(xn+1,yn+1)
E2(i,j) E3(i,j)
En(i,j) En+1(i,j)
En+2(i,j)
I(xn+2,yn+2)
16
MethodDefocus blur
……
……E1(i,j)
I(x1,y1) I(x2,y2) I(x3,y3)
I(xn,yn) I(xn+1,yn+1)
E2(i,j) E3(i,j)
En(i,j) En+1(i,j)
En+2(i,j)
I(xn+2,yn+2)
17
Method
◦Near-by object case
Focus on the flower
Focus on the flower
18
◦Near-by object case
Method Blur circle diameter
Objects in front of the
focus
Objects behind
the focus
Defocus Blur
Text synthesis inpainting
Blur shape as alpha map
Defocus Blur
Interpolation by alpha map
Shallow focus image
19
Experimental Result 1Defocus blur method proposed in the system
comparing with other blur filter results
Deep focus Shallow focus
20
Experimental Result 1Defocus blur method proposed in the system
comparing with other blur filter results
Gaussian blur Defocus blur Defocus blur + Bokeh
21
Experimental Result 1
22Defocus blur
Experimental Result 1
Gaussian blur Defocus blur + Bokeh
Original photograph
• Defocus blur method proposed in the system comparing with other blur filter results
23
Experimental Result 2with / without depth variation in the
background
Real photograph taken by DSLR
24
Experimental Result 2with / without depth
variation in the background
Original photograph
Result after post-processing
Depth variation
Depth fixed
25
Experimental Result 3
◦Near-by object case
Focus on the flower
Focus on the flower
26
◦Near-by object case
Without inpainting
Interpolation result
Experimental Result 3
27
Experimental Result 4Comparing with Photoshop
Photoshop (1 Hour) Our system (3~5min)
Original image
28
Applications - Privacy Preserving
29
Applications - Partial Viewing
30
Applications - Partial Viewing
31
Applications - Image Transition at Photo Browser
32
Applications - Image Transition at Photo Browser
33
Movie with defocus blur
effect
Applications - Image Transition at Photo Browser
34
Applications - Image Transition at Photo Browser
Concatenate two unrelated
images
35
ConclusionWe proposed an interactive refocusing
tool for background blurring◦ Simple user hint◦ Defocus blur kernel ◦ Concatenate two related picture
Future work◦ concatenate two or more unrelated
pictures Color based image retrieval technique
36
Q&A
37
Experimental Result 1Defocus blur method proposed in the system
comparing with other blur filter results
Deep focus Shallow focus Difference
38
Experimental Result 1Defocus blur method proposed in the system
comparing with other blur filter results
Shallow focus
Difference
Defocus blurPSNR:22.79
Gaussian blurPSNR:20.34
39
END
THANK YOU