A Robust Background Subtraction and Shadow Detection

Proc. ACCV'2000 , Taipei, Taiwan, January 2000.

井民全

Outline

• Introduction

• Background Modeling• Pixel Classification or Subtraction Operation• Automatic threshold Selection

• Experimental result

Introduction

Extracting moving objects from a video sequences

• Application• What’s problem with before?• Requirements• The purpose

Current Image

Background images

Training

Background

Moving object

The purpose1. Static background2. Using color information3. New color model

Ei (expected color)

R

G

B

Cdi=color distortion

Ii ( current color)

αi Ei

Color model

α>1 <1 =1

2)()( iiii EI || Ei ||

iii ICD

Brightness distortion

Background Modeling(training)

-A pixel is modeled by a 4-tuple <Ei,si, i ,bi> Ei = arithmetic means rgb value over n frame si = standard deviation of rgb value over n frame i = variation of the brightness distortion bi = variation of the chromaticity distortion

N

CDCDRMSbi

NRMSai

i

N

ii

i

N

ii

2

0

2

0

)()(

)1()(

)](),(),([

)]( ),( ),([

iiiSi

iiiEi

BGR

BGR

N=background frames

ai

-normalized color bands in the brightness distortion and chromaticity distortion.

222

2B

2G

2R

,,

2

)()(

)()(

)()(

)(

)()(I

)(

)()(I

)(

)()(I

)(

)()(min

ii

ii

ii

i

ii

i

ii

i

ii

i

iiI

B

B

G

G

R

R

B

B

G

G

R

R

BGRC c

ciCi

BGRc c

ciCi i

iiICD

,,

2

)(

)()(

Pixel Classification or Subtraction Operation

• Original background (B): brightness and chromaticity similar to the trained background.• Shaded background or shadow(S): similar chromaticity but lower brightness.• Highlighted background(H): similar chromaticity but lower brightness.• Moving foreground object(F): chromaticity different from from the expected values in trained background.

Different pixels yield different distributions of illumination and chromaticity distortion.

Using single threshold, we must do normalization

i

ii a

1

distortion brightness theofvariation

distortion brightnessˆ

i

i

b

CDCD

distortion chromacity theofvariation

distortion chromacity

otherwise :H

else 0, ˆ :

else ,ˆ and ˆ : B

else ,CD :

)( 12

i

ii

CD

S

iF

iM

Ei (expected color)

R

G

B

What’s problem of the dark pixel ?

distortion brightness

normalized for the bound low

otherwise :H

else 0, ˆ :

else ,ˆ and ˆ : B

else ˆor CD :

)(

lo

22

lo

where

S

iF

iMi

ii

iCD

Automatic threshold Selection

Total sample=NXYN=background frames

i

Freq.

Normalized Ahpha value0 +-

Fig. The normalized brightness distortion histogram

•The thresholds are selected according to the desired detection rate r

2 1

Automatic threshold Selection

Normalized CD iCD

Freq.

0 +

Fig. The normalized chromaticity distortion histogram

•The thresholds are selected according to the desired detection rate r

CD

Experimental result

Images size= 360 x 240Detection rate= 0.9999Lower bound of the normalized brightness distortion = 0.4

Fig. A sequence of an outdoor scene contain a person walking across the street

Fig. An application of the background subtraction in a motion capture system

Fig. game

Fig. An application of background subtraction in video editing

Conclusion

•Presented a background subtraction algorithm•Accurate, robust, reliable and efficiently computed•Real-time applications