color image segmentation
DESCRIPTION
Color Image Segmentation. Advisor : 丁建均 Jian-Jiun Ding Presenter : 蔡佳豪 Chia-Hao Tsai Date: 2011.03.17 Digital Image and Signal Processing Lab Graduate Institute of Communication Engineering National Taiwan University. Outline. Color image segmentation: Rough-set theoretic approach [1] - PowerPoint PPT PresentationTRANSCRIPT
Color Image Segmentation
Advisor : 丁建均 Jian-Jiun Ding
Presenter : 蔡佳豪 Chia-Hao Tsai
Date: 2011.03.17Digital Image and Signal Processing Lab
Graduate Institute of Communication Engineering
National Taiwan University
Outline
Color image segmentation: Rough-set theoretic approach [1]
Color-Based Image Salient Region Segmentation Using Novel Region Merging Strategy [2]
2DISP Lab, GICE, NTU
Flowchart
3DISP Lab, GICE, NTU
Histon
Visualization of color information for the evaluation of similar color regions in an image.
The segregation of the elements at the boundary, which can be applied in the process of image segmentation.
4DISP Lab, GICE, NTU
Histon
The histogram of the image I:
For a P×Q neighborhood around a pixel
I(m, n), the total distance
1 1
, , , for 0 1 and , ,M N
im n
g I m n i g g L i R G Bh
, , , ,Tp P q Q
m n d I m n I p qd
2 2 2, , , , , , , , , , , , , , ,d I m n I p q I m n R I p q R I m n G I p q G I m n B I p q B
5DISP Lab, GICE, NTU
Histon
A matrix X of the size M×N:
The histon:
1 , expanse,
0 otherwiseT m nd
X m n
1 1
, , , for 0 1 and1 , , ,M N
im n
g I m n i g g L i RX m G BH n
6DISP Lab, GICE, NTU
Roughness measure
The histogram and the histon can be correlated with the concept of approximation space in the rough-set theory.
The histogram value can be considered as the lower approximation and the histon value may be considered as the upper approximation.
7DISP Lab, GICE, NTU
Roughness measure
The vector of roughness measure:
The value of roughness is large (i.e. close to 1), this situation occurs in the object region where there is very little variation in the pixel intensities.
The value of roughness is small (i.e. close to 0), the variation in pixel intensities is near the boundary between the two objects.
1 , for 0 1 and = , ,i
ii
ghg g L i R G B
gH
8DISP Lab, GICE, NTU
Flowchart
Choose neighborhood (3 ×
3) and expanse (100).
Select significant peaks and valleys.
9DISP Lab, GICE, NTU
Thresholding
In general, the peaks in the histogram represent the different regions and the valleys represent the boundaries between those regions.
The peaks and valleys of the graph of roughness index versus intensity can also be used to segregate different regions in the image.
10DISP Lab, GICE, NTU
A example
(a) Original image, (b) segmented image based on histogram, (c) segmented image based on histon, (d) segmented image based on roughness index.
11DISP Lab, GICE, NTU
A example
(e) histogram of ‘red’ plane with peaks at 45, 72, and 254 and valleys at 56, and 209,
(f) histon of ‘red’ plane with peaks at 44, 72, and 254 and valleys at 56, and 210,
(g) roughness index of ‘red’ plane with peaks at 41, 75, 135, 161, and 249 and valleys at 56, 121, 144, and 215.
4472
254 135
7245
249161
25475
41
5620956 210 21514412156
12DISP Lab, GICE, NTU
How to obtain the significant peaks
Criterion 1:
The height of the peak > (the average value of roughness index for all the pixel intensities) ×1.2.
Criterion 2:
The distance between two peaks > 10. If the peak is satisfied the two criteria, it is
significant.13DISP Lab, GICE, NTU
How to obtain valleys
After the significant peaks are selected, the valleys are obtained by finding the minimum values between every two peaks.
14DISP Lab, GICE, NTU
Flowchart
15DISP Lab, GICE, NTU
Region merging
Obtaining clusters on the basis of peaks and valleys usually results in over-segmentation.
The clusters with pixels less than some predefined threshold are merged with the nearest clusters.(threshold= 0.1% of the total number of pixels in the image)
Two closest regions are combined to form a single region based on predefined distance between two clusters.(threshold= 20)
16DISP Lab, GICE, NTU
Segmentation results
17DISP Lab, GICE, NTU
Segmentation results
18DISP Lab, GICE, NTU
Segmentation results
(a) The original image, (b) histogram based approach, (c) histon based approach, (d) roughness index based approach.
19DISP Lab, GICE, NTU
Segmentation results(a) The original
image, (b) histogram
based approach,
(c) histon based approach,
(d) roughness index based approach.
20DISP Lab, GICE, NTU
Summarization
The number of histogram and histon peaks and valleys is the same and occur more or less at the same pixel intensities.
But, in the case of roughness histogram, we observe that we get additional peaks in all the R, G, and B components.
Therefore, roughness index based approach achieves better segmentation results. 21DISP Lab, GICE, NTU
Color-Based Image Salient Region Segmentation Using Novel Region Merging Strategy [2]
Introduction
The main purpose of this paper is not to precisely segment every single object in an image but to find the salient regions that are relatively meaningful to human perception.
For salient image segmentation, the salience is a macro property of an image. In other words, a salient region can be easily identified when we see an image.
23DISP Lab, GICE, NTU
The examples for salient regions
The dog and the grass are salient regions.
The herd of elephants is a salient region.
24DISP Lab, GICE, NTU
Flowchart(a) Dominant
color extraction.
(b) Region merging based on merging likelihood.
(c) Region merging based on color similarity.
25DISP Lab, GICE, NTU
The first phase: Dominant color extraction
Develop a new dominant color-extraction scheme based on nonparametric density estimation.
Given an n-dimensional dataset
the nonparametric density f(x)
where denotes unimodal density kernel
and σ is the bandwidth for the kernel.
; 1, , ,ni i Nx R
1
1( )
Ni
if x x xK
N
( )xK
26DISP Lab, GICE, NTU
The first phase: Dominant color extraction
Decompose the 3-D color space into three 1-D feature spaces, the nonparametric density is reformulated as
where h(r) denotes the histogram of an image for one of the three color channels, rk is the kth level of that channel and M is the total number of levels of it (In general, M=256).
1
1( ) , , ,
Mk i k i
if h K r Y U Vr r r r
M
27DISP Lab, GICE, NTU
The first phase: Dominant color extraction
After nonparametric estimation density, using the gradient ascent scheme we can easily find the local maxima. We select the local maxima of each channel and combine them to form the candidates of dominant colors.
28DISP Lab, GICE, NTU
The first phase: Dominant color extraction
(a) Original densities.
(b) Nonparametric densities.
(c) Color combinations.
29DISP Lab, GICE, NTU
The first phase: Dominant color extraction
A colory=25, u=220, v=148
y=67, u=123, v=151 (dominant color)
30DISP Lab, GICE, NTU
The first phase: Dominant color extraction
After the pixel assignments, each pixel in image has been replaced by the nearest candidate. Consequently, a quantized color image is obtained and a label map is created as well.
31DISP Lab, GICE, NTU
Flowchart(a) Dominant
color extraction.
(b) Region merging based on merging likelihood.
(c) Region merging based on color similarity.
32DISP Lab, GICE, NTU
The second phase : Region merging based on merging likelihood
Apply the region-growing algorithm on the label map of the quantized image to obtain initial regions.
Some of them may be very small and less important. Therefore, not all initial regions are salient.
In the following, we will define the salience of image region, and calculate the region importance. 33DISP Lab, GICE, NTU
The second phase : Region merging based on merging likelihood
Apply the region-growing algorithm on the label map of the quantized image to obtain initial regions.
Some of them may be very small and less important. Therefore, not all initial regions are salient.
In the following, we will define the salience of image region, calculate the region importance. 34DISP Lab, GICE, NTU
What are the salient regions?
Salient regions should be conspicuous.
Salient regions should be compact and complete.
35DISP Lab, GICE, NTU
The second phase : Region merging based on
Importance index and Merging likelihood Importance index: is used to measure the
importance of a region. Merging likelihood: is utilized to measure
the suitability of region merging. Whether a region should be merged mainly
depends on its “Importance index”, and where it should be merged into depends on the “Merging likelihood” between the region and each of its adjacent regions. 36DISP Lab, GICE, NTU
The second phase : Region merging based on
Importance index and Merging likelihood Importance index:
1
( )i ij j
ij
R Rij m
RRj
N NImp R
Max NN
37DISP Lab, GICE, NTU
Flowchart(a) Dominant
color extraction.
(b) Region merging based on merging likelihood.
(c) Region merging based on color similarity.
38DISP Lab, GICE, NTU
The second phase : Region merging based on
Importance index and Merging likelihood Merging likelihood: Color distance between regions. Boundary length between regions.
Region sizes of neighboring regions.
39DISP Lab, GICE, NTU
The second phase : Region merging based on
Importance index and Merging likelihood From the definition above, we can easily
find that a region with a smaller color distance, longer boundary length, and smaller region size will produce a higher value of merging likelihood.
40DISP Lab, GICE, NTU
Flowchart(a) Dominant
color extraction.
(b) Region merging based on merging likelihood.
(c) Region merging based on color similarity.
41DISP Lab, GICE, NTU
The third phase: Region merging based on color similarity
If the color distance between two connected important regions is less than Ts, they are similar and should be merged.
42DISP Lab, GICE, NTU
Segmentation results(a) Source image (b)Quantized image. (c) Initial regions. (d) Surviving regions represented in meancolors with region number.(e) Result (after further-merging) represented inmean colors. (f) Final Segmentation result.
43DISP Lab, GICE, NTU
Segmentation results(a) Source images. (b) Quantized images.(c) Segmentation results
represented in mean colors.(d) Segmentation results.
44DISP Lab, GICE, NTU
Segmentation results(a) Source images. (b) Quantized images.(c) Segmentation results
represented in mean colors.(d) Segmentation results.
Typical failure cases!
45DISP Lab, GICE, NTU
Summarization
Unlike the object segmentation, salient region segmentation is not necessary to extract each object in an image accurately but viewing the whole objects as a salient region.
Salient region segmentation is more feasible for applications such as region based image/video retrieval than is the object segmentation.
46DISP Lab, GICE, NTU
References
[1] Mushrif, M.M., A.K. Ray, “Color Image Segmentation: Rough-Set Theoretic Approach,” Pattern Recognition Letters, vol. 29, issue 4, pp. 483–493, March 2008.
[2] Y. H. Kuan, C. M. Kuo, and N. C. Yang, “Color-based image salient region segmentation using novel region merging strategy,” IEEE Trans. Multimedia, vol. 10, no. 5, pp. 832–845, Aug. 2008.
[3] R. C. Gonzalez, and R. E. Woods, "Chapter 10: Image Segmentation," Digital Image Processing 3rd Ed., pp.738-763, Prentice-Hall, 2008.
[4] S. Arora, J. Acharya, A. Verma, and Prasanta K. Panigrahi, "Multilevel thresholding for image segmentation through a fast statistical recursive algorithm," Pattern Recognition Letters, vol. 29, Issue 2, pp. 119-125, Jan. 2008.
[5] F. Yan, H. Zhang, and C. R. Kubeb, "A multistage adaptive thresholding method," Pattern Recognition Letters, vol. 26, Issue 8, pp. 1183-1191, June 2005.
47DISP Lab, GICE, NTU