7/30/2019 1569100457

1/7

Level Set Segmentation of Hyperspectral Images Using

Joint Spectral Edge and Signature Information

Radford R. Juang+ Philippe Burlina+* Amit Banerjee+

Johns Hopkins University Applied Physics Lab+ & Dept. of Computer Science*Laurel, MD 20723

radford.juang@jhuapl.edu

Abstract - This paper describes a new method for

segmenti ng hyperspectral imagery (HSI) using dynamic

curves. We are concerned about challenging HSI target

segmentation/detection use cases where the scene

includes confusers exhibiti ng a spectral return simil ar

to the desired signature and in cl ose proxi mity of the

object of in terest. Our method is based on a level sets

approach. I t fuses all available spectral bands and

incorporates spectral as well as spatial i nf ormati on to

obtain a fi ner target segmentation. The proposedmethod applies level set segmentati on to H SI by defi ni ng

an expansion force field that combines both

hyperspectral gradient information as well as the

desir ed spectral signature. We carry out experiments on

HSI datacubes obtained from a sensor spanning visible

and near I R and show improved resul ts when compared

to direct spectral matching in chal lenging close range

scenes including sign if icant l evel of nearby confusers.

Keywords: Level set segmentation, spectral signature,spectral gradient, hyperspectral imagery.

1 IntroductionRecent advances in hyperspectral sensors with high

spectral and spatial resolution have led to increased

interest in exploiting spectral imagery for target detection,

image segmentation, and object classification [1]

[4][6][7][13]. HSI provides spectral information for each

pixel in the image, resulting in a hyperspectral datacube.This spectrum often spans frequencies beyond the visible

range including near-, mid-wave, and long-wave infrared.

One advantage of the added spectral diversity is that it

provides rich information that may be exploited to yield abetter segmentation of the scene.

For our segmentation purposes, the image ismodeled as being composed of distinct, contiguous

regions, each of which is described by constant orhomogeneous attributes. Due to blurring and

additive/multiplicative noise, an image is a corrupted

version of an underlying piecewise smooth scene [11][17].

A natural approach to delineate the regions of an image is

to statistically estimate the attributes of the regions and

use the descriptions to differentiate between them. Many

such approaches to segmentation have been studied.

These methods can be broadly classified as either

Bayesian or data clustering approaches. Given an image,

these methods extract d-dimensional feature vectors from

the data and map them into a d-dimensional feature space.

Feature vectors from pixels of similar objects or materialswill form dense local clusters in the feature space.

Segmentation is then achieved by detecting and

segregating these clusters.

Bayesian methods generally proceed by formulating

statistical model assumptions for the region generation

and image formation processes. Maximum likelihood(ML) or maximum a posteriori (MAP) estimation is then

used for segmentation [9][16]. Techniques that modelimages as Markov random fields (MRFs) have been

extensively investigated [7][8].

A recent approach for segmenting hyperspectral

images using level sets was proposed by Ball and Bruce

[1][2][3]. Given a set of training signatures, they first

reduce the dimensionality of the datacube by selecting

best bands for classification. Next, they explore the use ofspectral angle, spectral information divergence, andmaximum likelihood as measures for classifyingsignatures into the corresponding classes. For each class,

they automatically determine threshold values by utilizing

the other classes as non-member data. Dimensionality

reduction may sometime compromise segmentation

results. That approach also requires that the training

signatures belong to at least two classes. Because they

have signatures from other classes, they can utilize thesesignatures as non-membership information to determine

the most optimal hyperplane that divides between those

classes and the target class signature(s).

In this paper, we describe a method which

incorporates spatial gradient information with spectral

measurements to achieve more accurate segmentationwith level sets. Specifically, we are interested in

obtaining a fine segmentation of an object given

information from the target signature only. Such anapproach obviates the need for knowledge about the

background classes. Hence, the proposed technique can

be considered as a single-class, instead of multi-class,

segmentation algorithm.

A common obstacle that emerges in single-class

problems is when the target signature is closely related to

that of the background, or when nearby/intersectingobjects in the scene have similar signatures. Our approach

leverages both the target spectral signature as well as edge

1927

7/30/2019 1569100457

2/7

information in a level set formulation to overcome this

obstacle.

The paper is organized as follows. In the next

section, level set methods are explained. We describe howlevel set methods can be introduced to resolve the oneclass problem of hyperspectral segmentation in Section 3.Finally, results of the approach are presented in Section 4.

2 Level Set SegmentationThe level set method is a mathematical formulation for

tracking moving fronts. Let ( )C t represent an evolving

contour at time t , and ( , , )F x y t represent a function

describing the force that drives the evolution of the curve

over time. One could naively pick random points along

( )C t and track their evolution independently

using ( , , )F x y t . However, there are a number of issues in

doing so including numerical stability problems. Osher

and Sethian devised a method to resolve this numerical

instability [15]. Instead of tracking the whereabouts of

points along ( )C t directly, they used a function ( , , )x y t where ( , , )x y t represents the signed distance of a point

( , )x y from the curve ( )C t . Points inside (resp. outside)

the curve would be assigned a negative (resp. positive)

distance. At any time, the curve ( )C t is represented by:

( ) {( , ) | ( , , ) 0}C t x y x y t = = (1)

Using , one can evolve the curve using the evolution

equation:

| | 0d

Fdt

+ = (2)

where F represents the speed function. This is called the

level set equation [15].The level set equation can be applied to the problem

of image segmentation, as demonstrated by Malladi et al.

[14], by defining F as a speed function dependent on the

curvature of the current curve, and/or on features from the

image. For example, define F as [14]:

( , ) ( )I A G

F x y k F F= + (3)

1

1 | ( , ) |Ik

G I x y=

+

(4)

2 2

2 2 3/ 2

( 2 )

( )

yy x x y xy xx y

G

x y

F

+=

+(5)

where ( , )I x y is the image, and | ( , ) |G I x y is the

magnitude of the gradient of the image after it is

convolved with a Gaussian kernel with variance , GF is

the curvature, and AF is a constant force term that causes

the curve to evolve in regions of little or no gradient. The

termI

k causes the curve to slow down and stop at edges,

while the curvature term AF causes the curve to preserve

its smoothness.

The level set method in the traditional form has its

own computational inefficiencies in maintaining

numerical accuracy. As is updated over time according

to (2), the function ( , , )x y t slowly departs from the

actual signed distance function. Osher and Sethians

original formulation required that be reinitialized

every so often, leading to a computationally expensive

formulation. Recent work by Li et al. in [12] pushed to

resolve these issues. They reformulated the level-setequation for segmentation to include a penalty term to

evolve so that it is close to the signed distance function.

Taking advantage of the signed distance property| | 1 = ,

Li et al. proposed the following functional as a metric to

characterize how close is to the signed distance

function [12]:

( )21

( ) 12

P dxdy

= (6)

2 \ and represents the domain of . Using this

metric, they proposed the following as the evolution

equation:

t

=

(7)

( ) ( ) ( )mP = + (8)

where 0 > is a weighting term that controls how

closely must follow the signed distance function, and

( )m is the function that drives the curves evolution to

the desired goals. They defined ( )m

as follows:

( ) ( ) ( )m g gL A = + (9)

where 0 > and are constants that control the penaltyrelated to the length and area of the evolved curve. Thelength and area of the evolved curve are controlled by:

( ) ( )g

L g dxdy

= (10)

( ) ( )g

A gH dxdy

= (11)where ( ) is the Dirac delta function and ( )H is the

Heaviside function. g is the force field that is derived

from the image data and characterizes how the curve will

1928

7/30/2019 1569100457

3/7

evolve. Putting all of these together, Lis formulation can

be summarized as the following:

| |

( ) ( )

| |

ddiv

dt

div g g

= +

+

(12)

3 Force FieldWe use the Li et al. [12] formulation to apply level set

segmentation to hyperspectral data. Their method not only

maintains a stable curve evolution without re-

initialization, it also eliminates the requirement that be

initialized as a signed distance function. These two factorssimplify and reduce the computational complexity of the

algorithm. We define g in equation (12) by fusing data

from all of the bands in the image. The design of ( )g

iscritical in the convergence to the desirable boundaries.First, we assume that an initial rectangular region of

samples,R , will be provided containing the samples that

we want the segmentation to match to. We define g as

follows:

1 2 1

1

( , ) ( , ) ( , )( , )

d x y d x y h x yg x y

M= (13)

where1

M is the maximum value of the numerator across

all ( , )x y . We combine information from several metrics

and normalize the result. The terms in the product aboveare defined as follows:

1 ,

2

1( , ) 1 ( , , )mahal x y R Rd x y D S

M= (14)

1

1 1

( ) ( )( , , )

T

mahalT T

u v u vD u v

u u v v

=

(15)

1( , )d x y incorporates the normalized Mahalanobis

distance measure between the vector at ( , )x y and the

mean vector of the samples in R .

,x yS is the spectrum at pixel ( , )x y .

R is the mean vector of the samples in R .

R is the covariance matrix for the samples in R .

2M is the maximum value of ,( , , )mahal x y R RD S across

all pixels.

Equation (15) is the normalized Mahalanobis distance

between vector u and v , where is the sample

covariance matrix. Next we have:

2 ,

3

1( , ) ( , , )

wcos x y Rd x y D S vM

= (16)

1

1 1( , , )

T

wcosT T

u vD u v

u u v v

=

(17)

R Rv = (18)

2 ( , )d x y includes the whitened cosine of the angle

between the image vector at ( , )x y and the re-centered

mean of the samples in R . R is a scalar containing the

average (across the bands) of the vector elements inR

.

This makes v the zero-centered version of R . 3M is the

maximum value of,

( , , )wcos x y R

D S v across all ( , )x y .

1( , )h x y is defined by:

1

4

| ( , ) |( , ) 1

H x yh x y

M

=

(19)

This is a map of the edges where the edge values are

small/close to zero. The operation H is a gradient as

defined by Stokman and Gevers [18]. 4M is the

maximum value of | ( , ) |H x y across all ( , )x y .

To improve the results ofg, we can also threshold

each of the terms in g so as to place emphasis on the

higher values. We define a function ( , )x t as follows:

( , )

0

x t x tx t

x t

=

(23)

Where 1t , 2t , 3t are threshold parameters that can be

used to tune the algorithm,2

M is the maximum value of

1 1( ( , ), )d x y t across all ( , )x y , and 3M is the maximum

value of2 2

( ( , ), )d x y t across all ( , )x y . The greater than

operatorx y> returns 1 forx y> and 0 otherwise.

1929

7/30/2019 1569100457

4/7

3.1 Automatic Threshold SelectionGiven , the maximum ratio of the number of pixels inthe target to segment to the number of pixels in the image(i.e. the maximum size ratio of the target image to

segment), we describe here how to automatically tune thethreshold value(s) for a field ( , )F x y . We apply the

algorithm in Table 1 to 1( , )d x y , 2 ( , )d x y , 1( , )h x y , and

| ( , ) |H x y to determine 1t , 2t , and 3t . To express this in

written form, we replace ( , )x t in equations (21)-(23)

with '( , )x :

( , ) ( , ( , ))GetThresholdValuex x x = (24)

1 1

1

2

( ( , ), )( , )

d x yd x y

M

=

(25)

2 2

2

3

( ( , ), )( , )

d x yd x y

M

=

(26)

[ ]1 3( , ) 1 (| ( , ) |, ) 0h x y H x y = > (27)

Our overall algorithm is listed in Table 2:

LevelSetSegment(I,R , 1 , 2 , 3 ):

1 Construct a force field g :a. Compute metrics 1 2 1, ,d d h as

described in equations (24)-(27).

b. Compute:1 2 1

1

( , ) ( , ) ( , )d x y d x y h x yg

M

=

Where 1M is the maximum of the

numerator in g

2 Apply the level set evolution

algorithm as described in [12] using the

field g. We keep evolving until growth

of the enclosed area in the curve is

trivial. We use the rectangle R as theinitial contour and evolve our curve

outwards.

Table 2 - Overall algorithm for level set segmentation of

a one class problem

4 ResultsWe provide results of applying our level set algorithm

with and without gradient information and compare these

with direct spectral matching (see Figure 1 to Figure 3).

These datacubes were taken with a hyperspectral camera

that captures a scene with 22 spectral bands across thewavelengths 400.00 nm - 1000.00 nm. Each image has aresolution of 512x400 pixels.As we argued earlier, one of the challenges we are

addressing in this paper lies in trying to differentiate

between the target and neighboring objects with similarspectra. For example, the spectral signature for the shirt

and pants of the person in Figure 1 are quite similar as

seen in the plot in Figure 4.

If we were to rely solely on spectral matching, suchas the whitened cosine distance between spectra, the

segmentation result would be highly sensitive to the

selected threshold. Incorrect selection of the thresholds, as

shown in the matching and thresholding results in figures

1c and 2c, leads to poor segmentation as objects of similarspectra act as confusers.

The advantage of using level set methods is that we

can be more permissive in the selection of the matching

threshold. Consider Figures 1c and 2c for which straightmatching and thresholding yields significant errors.

However, for similar threshold values, we can apply the

level set method described above and dynamically grow

curves over the region of interest. The curves growth is

varied according to the thresholded whitened cosine

distance to the target signature. We see in figures 1a,b and

2a,b that the results are significantly improved over thedirect matching and thresholding results.

GetTresholdValue(F, ):1 Construct a normalized histogram of

F across all ( , )x y .

2 Construct a normalized cumulative

distribution function (CDF) by

cumulatively adding each bin of

the histogram (running sum).

3 Find the first value in the CDF

that is greater than 1 and returnthe value of the bin corresponding

to the value.

Table 1 Algorithm to automatically compute the

thresholds from a field ( , )F x y

1930

7/30/2019 1569100457

5/7

We show that in some circumstances it is beneficial to

incorporate gradient information (as described in Section3) to differentiate the region of interest from neighboring

confusers with similar spectral signatures. The scenario in

Figure 1 includes two people crossing with shirts that are

spectrally similar. Furthermore, the shirt and pants of theleft person in Figure 1 also have similar spectra (as

illustrated in Figure 4). Without spectral edge information,

as shown in Figure 1b, the level set methods evolve thecurve into the neighboring regions of the pants and the

crossing persons shirt. If we incorporate gradient

information (as described in Section 3), it can be seen

from Figure 1a that the problem is avoided. In figure 2a,

the spectral edges have a lesser impact since bordering

objects do not share the same spectral characteristics as

the target object.

5 ConclusionIn this paper, we have shown how to incorporate the level

set method into the problem of hyperspectral datacube

segmentation where only one class of signatures (the

target class) is known. We provide an algorithm whichrelies on a force field that incorporates both spectral

signature and spectral edge information and we show how

it improves the segmentation results when compared to

direct matching.

Figure 1 - (a) Left: Segmentation on HSI image using level sets with spectral edge information. (b) Middle:Segmentation on HSI image using level set without spectral edge information. (c) Right: Segmentation on HSI image viadirect spectral matching and thresholding. All results are obtained for same thresholding value.

Figure 2- (a) Left: Segmentation on HSI image using level sets with spectral edge information. (b) Middle:

Segmentation on HSI image using level set without spectral edge information. (c) Right: Segmentation on HSI image via

direct spectral matching and thresholding. All results are obtained for same thresholding value.

1931

7/30/2019 1569100457

6/7

Figure 4 - Plot of the mean spectra signature for shirts andpants of person in Figure 1

References

[1] J.E. Ball and L.M. Bruce, Level Set Segmentationof Remotely Sensed Hyperspectral Images. Proceedings

of the 2005 IEEE International Geoscience and Remote

Sensing Symposium, vol. 8, pp. 5638--5642, July, 2005.

[2] J. E. Ball and L.M. Bruce, Level Set HyperspectralSegmentation: Near-Optimal Speed Functions using Best

Band Analysis and Scaled Spectral Angle Mapper, in

Geoscience and Remote Sensing Symposium, 2006, pp.2596--2600, July, 2006.

[3] J.E. Ball and L.M. Bruce, Level Set HyperspectralImage Classification Using Best Band Analysis, inIEEE

Transactions on Geoscience and Remote Sensing, vol.45,

no.10, pp.3022--3027, October, 2007.

[4]A. Banerjee, P. Burlina, C. Diehl, Support vectormethods for anomaly detection in hyperspectral imagery,inIEEE Transactions on Geoscience and Remote Sensing,

vol. 44, no. 8, pp. 2282--2291, August, 2006.

[5]

A. Banerjee, P. Burlina, J. Broadwater, A machinelearning approach for finding hyperspectral End-members, in Trans.IGARS, Barcelona, 2007.

[6] S.G. Beaven, D. Stein, and L.E. Hoff, Comparisonof Gaussian mixture and linear mixture models for

classification of hyperspectral data, in Proc. IGARSS2000, Honolulu, HI, pp. 1597--1599, July, 2000.

[7] C.A. Bouman, and M. Shapiro, A MultiscaleRandom Field Model for Bayesian Image Segmentation,

IEEE Trans. Image Processing, Vol. 3, pp. 162-177,

1994.

[8] S. Geman and D. Geman, Stochastic Relaxation,Gibbs Distribution, and Bayesian Restoration of Images,

IEEE Trans. Pattern Anal. Machine Intell., Vol. 6, pp.721-741, 1984.

[9] P.A. Kelly, H. Derin, and K.D. Hartt, AdaptiveSegmentation of Speckled SAR Images Using a

Hierarchical Random Field Model, IEEE Trans.

Acoustics, Speech, and Signal Processing, 1988.

Figure 3 Result from different iterations of the curve evolution.

1932

7/30/2019 1569100457

7/7

[10] H. Kwon and N. Nasrabadi, Kernel RX-algorithm:A nonlinear anomaly detector for hyperspectral imagery,

inIEEE Transactions on Geoscence and Remote Sensing,

vol. 43, no. 2, pp. 388--397, February, 2005.

[11] Y. LeClerc, Constructing Simple StableDescriptions for Image Partitioning, International

Journal of Computer Vision, Vol. 3, pp. 73-102, 1989.

[12] C. Li et al., Level Set Evolution without Re-Initialization: A New Variational Formulation, CVPR,

2005, vol. 1, pp. 430--436, June, 2005.

[13] X. Lu, L.E. Hoff, I.S. Reed, M. Chen, and L.B.Stotts, Automatic target detection and recognition inmultiband imagery: A unified ML detection and

estimation approach,IEEE Trans. Image Processing, vol.

6, pp. 143--156, January, 1997.

[14] R. Malladi, J.A. Sethian, B.C. Vemuri, Shapemodeling with front propagation: a level set approach, in

IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 17, no. 2, pp. 158--175, February, 1995.

[15] S. Osher and J. Sethian, Fronts propagating withcurvature-dependent speed: Algorithms based onHamilton-Jacobi equations, Journal of Computational

Physics, vol. 79, pp. 12--49, 1988.

[16] E. Rignot and R. Chellappa, Segmentation ofPolarimetric Synthetic Aperture Radar Data,IEEE Trans.

Image Processing, Vol. 1, pp. 281-300, 1992.

[17] A. Rosenfeld, R. Hummel, and S. Zucker, SceneLabeling by Relaxation Operations, IEEE Trans.Systems, Man, and Cybernetics, Vol. 6, pp. 420-433,

1976.

[18] H. Stokman and T. Gevers, Detection andClassification of Hyper-Spectral Edges,Proc. 10th British

Machine Vision Conf., pp. 643--651, 1999.

1933