電腦視覺 computer and robot vision i

1

電腦視覺Computer and Robot Vision

I

Chapter2: Binary Machine Vision:

Thresholding and Segmentation

Instructor: Shih-Shinh Huang

2

Contents

Introduction

Thresholding

Connected Components Labeling

Signature Segmentation and Analysis

3

Computer and Robot Vision I

2.1 Introduction

Introduction

4

Binary Machine Vision

Binary Image Binary Value 1: Part of Object Binary Value 0: Background Pixel

Definition of Binary Machine Vision Generation and analysis of such a binary image

2.1 Introduction

5

Binary Machine Vision

Thresholding It is the first step of binary machine vision It is a labeling operation

Connected Components / Signature Analysis They are multilevel vision grouping techniques. They make a transformation from image pixels to

more complex units.• Regions

• Segments

2.1 Introduction

6

Computer and Robot Vision I

2.2 Thresholding

Introduction

7

Introduction

What is Thresholding ? It is a labeling operation. It assigns a binary value to each pixel.

• Binary Value 1: pixels have higher intensity values

• Binary Value 0: pixels have higher intensity values

2.2 Thresholding

128T

8

Introduction

Mathematical Formulation

: row and column

: gray-level intensity image

: intensity threshold

: binary intensity image

2.2 Thresholding

TcrIifTcrIif

crB),(0),(1

),(

),( cr

(.,.)I

(.,.)B

T

9

Introduction2.2 Thresholding

(.,.)I

5T

(.,.)BHow to select an appropriate threshold ?

10

Introduction

Approaches Global Thresholding: use a global value to make the pixel

distinction in the image.

Local Thresholding: use spatial varying threshold to label

the local pixels.

2.2 Thresholding

T

Image Image

1T 2T

3T 4T

11

Histogram

Definition of Histogram

Histogram Probability

2.2 Thresholding

}),(|),{(#)( mcrIcrmh

(.)h

number of elements m spans each gray level value e.g. 0 - 255

255,...,1,0:)( IIP

CRIhIP

)()(

12

Histogram

Examples

2.2 Thresholding

)(mh

)(mh

13

Histogram2.2 Thresholding

)(mh

Histogram

14

2.2 Thresholding

T=110 T=130 T=150 T=170

15

Within-Group Variance

Observations A group is a set of pixels with intensity homogeneity. Homogeneity is measured by the use of variance

• High homogeneity group has low variance

• Low homogeneity group has high variance

Objective Select a dividing score such that the weighted sum

of the within-group variances is minimized.

2.2 Thresholding

16

Within-Group Variance

Definition: weighted sum of group variances

: probability for the group with values

: probability for the group with values : variance for the group with values : variance for the group with values

)()()()()( 222

211

2 ttqttqtW

)(22 t

)(21 t

)(1 tq

)(2 tq

t

t

t

t

2.2 Thresholding

17

Within-Group Variance

Objective Formulation

Find a threshold which minimizes

2.2 Thresholding

*t )(2 tW

)(minarg 2* tt W

t

i

iPtq1

1 )()(

t

i

tqiiPt1

11 )(/)()(

I

ti

iPtq1

2 )()(

I

ti

tqiiPt1

22 )(/)()(

)(/)()]([)( 12

11

21 tqiPtit

t

i

)(/)()]([)( 22

222 tqiPtit

I

iti

18

Within-Group Variance

Implementation Issue Step1: For t=0,…,255

Step2: Compute , , , and

Step3: Compute

Step4: If is less than the value in the

previous iteration

2.2 Thresholding

)(22 t)(21 t)(1 tq )(2 tq

)()()()()( 222

211

2 ttqttqtW

)(2 tW

)(2 tW

tt *

All variables should be re-compute at each iteration.

19

Within-Group Variance

Implementation Issue Speed-Up Formulation

2.2 Thresholding

I

i

iPi1

22 )()(

T

i

iiP1

)(

T

ti

t

i

iPttiiPtti1

222

1

211

2 )())()(()())()((

.......)())(())())(((2))((1

2111

21

2

t

i

iPtttiti

20

Within-Group Variance

Implementation Issue Speed-Up Formulation

2.2 Thresholding

.......)())(())())(((2))((1

2111

21

2

t

i

iPtttiti

t

i

iPtti1

11 0)())())(((

t

i

iPtti1

22

22 )())(())((

t

i

iPtti1

21

21

2 )())(())((

21

Within-Group Variance

Implementation Issue Speed-Up Formulation

2.2 Thresholding

)(])([)())(( 12

11

21

2 tqtiPtit

i

)(])([)())(( 22

21

22 tqtiPti

I

ti

)()()()( 222

211

2 ttqttq

2222

11 ])()[(])()[( ttqttq

22

Within-Group Variance

Implementation Issue Speed-Up Formulation

2.2 Thresholding

)()()()( 222

211

2 ttqttq

2222

11 ])()[(])()[( ttqttq

2122

1122 ])()[(])()[( ttqttqw

)]()()][(1)[( 211122 tttqtqw

)()()()( 2211 ttqttq

23

Within-Group Variance

Implementation Issue Speed-Up Formulation

2.2 Thresholding

)]()()][(1)[( 211122 tttqtqw

constant minimize maximize

)]()()][(1)[(maxarg 2111* tttqtqt

24

Within-Group Variance

Implementation Issue Speed-Up Formulation

• We have recursive form to compute optimal threshold.

2.2 Thresholding

)]()()][(1)[(maxarg 2111* tttqtqt

)1()()1( 11 tPtqtq

)1()1()1()()()1(

1

111

tqtPtttqt

25

Within-Group Variance

Example

2.2 Thresholding

26

Kullback Information Distance

Assumption The observations come from a weighted mixture of

two Gaussians distributions.

• Gaussian Distribution of Background

• Gaussian Distribution of Object

2.2 Thresholding

),( 211 u

),( 222 u

2

2

22

1

1 )(21

2

2)(

21

1

1

22)(

ii

eqeqif

27

Kullback Information Distance2.2 Thresholding

),( 211 u ),( 2

22 u

Background Gaussian Distribution

ObjectGaussian Distribution

28

Kullback Information Distance

Objective Formulation Determine a threshold T that results in two Gaussian

distributions which minimize Kullback divergence

• P(I) : observed histogram distribution

• f(I) : a mixture of Gaussian distributions determined by T

2.2 Thresholding

])()(log[)(

1 ifiPiPJ

I

i

2

2

22

1

1 )(21

2

2)(

21

1

1

22)(

ii

eqeqif

29

Kullback Information Distance

Objective Formulation Known Parameter: Observed Histogram

Unknown Parameter: Two Gaussian Distributions

2.2 Thresholding

)(),...2(),1( IPPP

(.)P

2

2

22

1

1 )(21

2

2)(

21

1

1

22)(

ii

eqeqif

),,(),,,( 222111 qq

30

Kullback Information Distance

Solution Derivation

2.2 Thresholding

])()(log[)(

1 ifiPiPJ

I

i

)(log)(log)(1

ifiPiPI

i

)(log)()(log)(11

ifiPiPiPI

i

I

i

Constant

31

Kullback Information Distance

Solution Derivation

Assumption: The modes are well separated.

2.2 Thresholding

)(log)(minarg])()(log[)(minarg

11

ifiPifiPiP

I

i

I

i

tieq

tieq

if i

i

2

2

2

2

1

1

)(21

2

2

)(21

1

1

2

2)(

32

Kullback Information Distance

Solution Derivation

2.2 Thresholding

)(log)()(1

ifiPtHI

i

2

2

22

1

1 )(21

2

2

1

)(21

1

1

1 2)(

2)()(

iI

ti

it

i

eqiPeqiPtH

2211 loglog22log1)( qqqqtH

2211 loglog21 qq

33

Kullback Information Distance

Implementation Issue Step1: For t=0,…,255

Step2: Compute , , , and

Step3: Compute

Step4: If is less than the value in the

previous iteration

2.2 Thresholding

)(22 t)(21 t)(1 tq )(2 tq

)(tH

tt *

)(log)()(1

ifiPtHI

i

)(tH

34

Kullback Information Distance

Example

2.2 Thresholding

35

Kullback Information Distance2.2 Thresholding

Within Group Variance (Otsu)

Kullback Information (Kittler-Illingworth)

36

Computer and Robot Vision I

2.3 Connected Component Labeling

Introduction

37

Introduction

Description Connected Components labeling is a grouping

operation.

It performs the unit change from pixel to region or segment.

All pixels are given the same identifier • Have value binary 1

• Connect to each other

2.3 Connected Component Labeling

38

Introduction

Terminology label: unique name or index of the region connected components labeling: a grouping

operation pixel property: position, gray level or brightness

level region property: shape, bounding box, position,

intensity statistics

2.3 Connected Component Labeling

39

Connected Component Operators

Definition of Connected Component

Two pixels and belong to the same connected component if there is a sequence of 1-pixels , where

•

•

• are neighbor

2.3 Connected Component Labeling

qpC),...,( 10 nppp

pp 0qpn

nipp ii ,...,1:,1

40

Connected Component Operators

Neighborhood Types

2.3 Connected Component Labeling

4-connected

8-connectedOriginal Image Connected

Components

41

Connected Component Algorithms

Common Features Process a row of image at a time Assign a new labels to the first pixel of each

component. Propagate the label of a pixel to its neighbors to

the right or below it.

2.3 Connected Component Labeling

42

Connected Component Algorithms

Common Features

2.3 Connected Component Labeling

• What label should be assigned to A• How does the algorithm keep track of the

equivalence of two labels• How does the algorithm use the equivalence

information to complete the processing

43

Algorithm1: Iterative Algorithm

Algorithm Steps Step1 (Initialization): Assign an unique label to

each pixel.

Step2 (Iteration) : Perform a sequence of top-down and bottom-up label propagation.

2.3 Connected Component Labeling

• Use no auxiliary storage• Computational Expensive

44

Algorithm2: Classic Algorithm

Two-Pass Algorithm Pass 1:

• Perform label assignment and label propagation• Construct the equivalence relations between labels

when two different labels propagate to the same pixel.

• Apply resolve function to find the transitive closure of all equivalence relations.

Pass 2:

• Perform label translation.

2.3 Connected Component Labeling

45

Algorithm2: Classic Algorithm

Example:

2.3 Connected Component Labeling

{2=4}{3=5}{1=5}

46

Algorithm2: Classic Algorithm

Example: Resolve Function

2.3 Connected Component Labeling

{2=4}{3=5}{1=5} {2=4}{1=3=5}

• Computational Efficiency • Need a lot of space to store equivalence

47

Computer and Robot Vision I

2.4 Signature Segmentation and

Analysis

Introduction

48

Introduction

Description Signature analysis perform unit

change from the pixel to the segment. It was firstly used in character

recognition Definition of Signature

The signature, which is a projection, is the histogram of the non-zero pixels of the masked image.

2.4 Signature Segmentation and Analysis

49

Introduction

General Signatures Vertical Projection Horizontal Projection Diagonal Projection

2.4 Signature Segmentation and Analysis

},1),(|),{(#)( rxyxyxrh

},1),(|),{(#)( cyyxyxrv

50

Signature Segmentation

Steps Thresholding: generate the binary image.

Projection Computation: compute the vertical, horizontal, or diagonal projections.

Projection Segmentation: divide the image into several segments or regions according to the signatures.

2.4 Signature Segmentation and Analysis

51

Signature Segmentation2.4 Signature Segmentation and Analysis

52

Signature Segmentation2.4 Signature Segmentation and Analysis

53

Signature Segmentation2.4 Signature Segmentation and Analysis

OCR: Optical Character Recognition

MICR: Magnetic Ink Character Recognition

The End

Computer and Robot Vision I