HCI Project :An Iterative Optimization Approach for

Unified Image Segmentation and Matting

組員 : P78961304 周智倫 P76961023 黃琮聖 P76974157 蔡偉民 P76974482 鄭世鴻

Abstract

• Extracting a matte by previous approaches require the input image to be pre-segmented into three regions (trimap).

• This pre-segmentation based approach fails for images with large portions of semi-transparent foreground.

• In this paper we combine the segmentation and matting problem together and propose a unified optimization approach based on Belief Propagation.

Introduction

• The observed image I(z) (z = (x, y)) is modeled as a linear combination of foreground image F(z) and background image B(z) by an alpha map: I(z) = αzF(z) + (1 − αz)B(z)

• Image Matting– estimating an opacity (alpha value) and

foreground and background colors for each pixel in the image.

Limitations of a Trimap

• To generate good mattes, all these approaches require the user to ”carefully” specify the trimap.

• it is almost impossible to manually create an optimal trimap.

Limitations of a Trimap (cont.)

• Automatically generated trimaps based on the binary segmentation result is non-optimal, since it always has uniform thickness regardless of local image characteristics.

MRF Construction

• Each pixel in and are treated as a node in the MRF• Minimize the total energy of the following function

• ： How well the estimated alpha value , and foreground and background color for fit with the actual color

• ： The smoothness energy which penalizes inconsistent alpha value changes between two neighbors and

cU cU~

p qp

qpspd VVV,

),()(

dV pp

pC

sVp q

Markov Random Field• 上層 – 原值• 下層 – 估計值

Predefined arguments

• Discretize the possible alpha value to 25 levels between 0 and 1,denoted as , k=1,…,25

• Each level corresponds to a possible state for a node in the MRF

• The local neighborhood area is defined to have a radius of r=20

in detail

• Compute the likelihood of each alpha level as

• The set of valid foreground samples, are then weighted by their uncertainty and distance, by

dV

K

k k

kkpd

pL

pLV

1)(

)(1)(

k

)2/))1(,(exp(.1

)(22

1 12

kd

pj

kpi

kpc

N

i

N

j

Bj

Fik BFCdww

NpL

))),(

exp()))(1(2

2

w

ii

Fi

ppspuw

in detail

• The smoothness cost is defined as

sV

)/)(exp(1),( 222121 ssV

Belief Propagation Optimization

• Use loopy belief propagation (BP) to solve problem– Finding a labeling with minimum energy corresponds to

the MAP estimation problem

• It works by passing messages along links in the constructed path

BP Algorithm(1)

• In each iteration, new messages are computed for each possible state

• H (p) \ q denotes the neighbors of p other than q• c is a normalization factor

BP Algorithm(2)

• After T iterations a belief vector is computed for each node

• The state the maximizes at each node is selected as the estimated level

k p

kb p

pk p *

p

BP Algorithm(3)

• If =1, set the color as a new foreground sample

• If =0, set the color as a new background sample

• Otherwise, choose the pair of foreground and background colors from the group of samples

*

p C p

*

p C p

BP Algorithm(4)

• Then, the uncertainty value u(p) is updated as

• and are weights for the selected pair of foreground and background samples w

F

i

*

wB

i

*

Iterative Belief Propagation for Image Matting

Extension to Video

is small as definite foreground

Result and Comparisons

Propose approach

Bayesian

Extracted foreground and novel background

Foreground and background is similar

Not work well

Start with initial trimap

Rough trimap the user created

Bayesian

Proposedapproach

Summary and conclusion

• Proposed a approach to solve image matting problem

• combines the problems of segmentation and matting into a unified formula

• Does not require a well specified trimap