research and implementation of smoke detection in video streams [email protected]

of 61

Research and Implementation

of

Smoke Detection in Video Streams

Naveed Akram 内维德

School of Computer Science and Engineering ,

Beihang University, Beijing

[email protected]

of 61

Agenda

Introduction of Research Work

Background and Motivation

Overview of Research Work

Research and Implementation

Results / Demo

Question / Answer

of 61

Introduction

An image processing based technique

is proposed to detect fire smoke in

video streams.

Basic Idea is to use already installed

CCTV cameras for smoke detection

instead of using conventional smoke

detectors.

of 61

Background and Motivation

Fire is one of the biggest disasters for

the human beings.

In 2009 (only in USA)

◦ estimated 1,348,500 fires

◦ 3,010 deaths

◦ 17,050 injuries

◦ $12.5 Billion property loss

Backgro

und a

nd

Mo

tiva

tion

of 61

Why we need this study

Traditional methods can not work in

some situation and fail to detect fire

smoke.

◦ Some times can not detect at all.

◦ Produces delay and need close proximity

◦ Fail in open places, outdoor, forests

◦ No method to verify false alarms

We are proposing a method that can

overcome these issues.

Backgro

und a

nd

Mo

tiva

tion

of 61

Video Based Fire Detection

System Lower cost

Faster response

Large coverage area

Verification of false alarms

Backgro

und a

nd

Mo

tiva

tion

of 61

Challenges

Still evolving technology

Difficult to process due to variability in

smoke density, lighting, diverse

background, interfering non-rigid objects

etc.

Primitive image features such as

intensity, motion, edge, and obscuration

do not characterizes smoke very well in

the videos

Visual pattern of smoke is difficult to

model.

Backgro

und a

nd

Mo

tiva

tion

of 61

Related Work

In recent literature a number of

methods for smoke detection in videos

are presented based on

◦ Self-Similarity

◦ Motion and optical flow

◦ Wavelet Transformation

(flickering/∆energy)

◦ Based on Color models

◦ Night Vision fire detection

◦ Feature’s based

Backgro

und a

nd

Mo

tiva

tion

of 61

OVERVIEW OF RESEARCH WORK

of 61

Overview of Research Work

Pre-

Processing

Moving

Target

Detection

Feature

ExtractionSmoke

Detection

Ove

rvie

w o

f Re

se

arc

h

Work

of 61

Pre Processing

Pre-

Processing

Moving

Target

Detection

Feature

ExtractionSmoke

Detection

1.Frame Extraction from Video Stream

2.Color to gray scale conversion

3.Median filtering

Ove

rvie

w o

f Re

se

arc

h

Work

of 61

Moving Target Detection

Moving

Target

Detection

Feature

ExtractionSmoke

Detection

Pre-

Processing

1.Background Subtraction

2.Grayscale to Binary Conversion

3.Contour Extraction

Ove

rvie

w o

f Re

se

arc

h

Work

of 61

Feature Detection

Feature

ExtractionSmoke

Detection

Pre-

Processing

Moving

Target

Detection

1.Calculation of static and dynamic features of

moving target object.

2.Such as Local Wavelet Energy, Growth rate,

Disorder, flickering frequency, Source

Stability etc.

Ove

rvie

w o

f Re

se

arc

h

Work

of 61

Smoke Detection

Feature

ExtractionSmoke

Detection

Pre-

Processing

Moving

Target

Detection

1.Training of Neural Network (Once)

2.Use of Neural Network to decide either

smoke or not

Ove

rvie

w o

f Re

se

arc

h

Work

of 61

RESEARCH AND IMPLEMENTATION

of 61


Moving

Target

Detection

Feature

ExtractionSmoke

Detection

Background

SubtractionContour

Extraction

Researc

h a

nd

Imple

me

nta

tion

of 61

Background Subtraction

Preliminary frame differencing

Dynamics Matrix

Adaptive Background Update

Researc

h a

nd

Imple

me

nta

tion

of 61


Background Update Model

of 61


Foreground detectionStart

Current Filtered

Frame, I(k)

B(i,j) from Background

update Model

If FD(i,j) >TForeground

FG(i,j)=0

Foreground

FG(i,j)=I(i,j)

.F.

.T.

Link to Next

Process

Absolute Frame Difference

, , ,( ) ( ) ( )i j i j i jFD k I k B k

Researc

h a

nd

Imple

me

nta

tion

of 61


Results

(a) Original Frames (b) Foreground Objects

Researc

h a

nd

Imple

me

nta

tion

of 61

Contour Extraction

Grayscale to Binary Conversion

(Otsu's method )

Dilation and Erosion of Binary Image

Researc

h a

nd

Imple

me

nta

tion

of 61

Grayscale->Binary

Conversation

Researc

h a

nd

Imple

me

nta

tion

of 61

Erosion and Dilation

Erosio

n

Dilation

Researc

h a

nd

Imple

me

nta

tion

of 61

Contour Extraction

Erosio

n

Dilation

Contour= Dilation-

Erosion

Researc

h a

nd

Imple

me

nta

tion

of 61

Results of Moving Target

Detection

GR=4.07 DS=0.01 NOB=2 FLC=47.0 EN=-1.0 SS=1394

Smoke Detected Fr 252 Fgnd.Fr 221

Researc

h a

nd

Imple

me

nta

tion

of 61

Feature Extraction

Pre-

Processing

Moving

Target

Detection

Smoke

Detection

Dynamic

FeaturesStatic

Features

Feature

Extraction

Growth Rate DisorderNumber of

Segments

Source

Stability

Flickering

Frequency

Local

Wavelet

Energy

Researc

h a

nd

Imple

me

nta

tion

of 61

Growth Rate

as percentage change in Area of the

current frame with reference to

previous frame

( , ) Number of '1's in binary i

A x y image i

1

1

( , ) ( , ) 2

( , )

i i

i

A x y A x yGrow thRate i

A x y

Researc

h a

nd

Imple

me

nta

tion

of 61

Growth Rate Results

Researc

h a

nd

Imple

me

nta

tion

of 61

Disorder Feature

Smoke has another feature that

makes it distinguish from other

foreground objects that is its rapidly

changing shape. This feature of

smoke is called disorder

Researc

h a

nd

Imple

me

nta

tion

of 61

Disorder Feature

Human

Movement

Smoke

Video

Researc

h a

nd

Imple

me

nta

tion

of 61

Results of

Disorder Feature

280 285 290 295 300 305 310 315 320 325 3301.6

1.8

2

2.2

2.4

2.6

2.8

3Disorder VS Frame No

Frame No

Dis

ord

er

280 285 290 295 300 305 310 315 320 325 3303.5

4

4.5

5

5.5

6Disorder VS Frame No

Frame No

Dis

ord

er

Human

Movement

Smoke

Video

Researc

h a

nd

Imple

me

nta

tion

of 61

Number of Segments

While smoke spreads it splits into

different small / large patches.

Sometime these patches may

increase to 8 to 10.

We used 8-connected pixels algorithm

to calculate number of segments in

current video frame.

Researc

h a

nd

Imple

me

nta

tion

of 61

Number of Segments

Researc

h a

nd

Imple

me

nta

tion

of 61

Frequent Flickering

A pixel at the edge of a turbulent flame

or boundary of smoke could appear

and disappear several times in one

second of a video sequence. This kind

of temporal periodicity is commonly

known as flickering

Researc

h a

nd

Imple

me

nta

tion

of 61

Frequent Flickering

Transition Matrix

Frequency Matrix

Pixels on contour

Researc

h a

nd

Imple

me

nta

tion

of 61

Frequent Flickering

Thresholding

Feature Calculation

Researc

h a

nd

Imple

me

nta

tion

of 61

Researc

h a

nd

Imple

me

nta

tion

Flow Chart

of 61

Frequent Flickering

Subtracting consecutive frames to get transition

matrices

Frequency MatrixAND with ContourThresholding

Researc

h a

nd

Imple

me

nta

tion

of 61

Results

Sm

oke

Vid

eo

Hum

an

Movem

ent V

ideo

of 61

Local Wavelet Energy

Sharp edges in the background are

sources of high frequency and hence

high wavelet energy

Fire smoke can smoothen the edges

in an image because of the fuzzy

effect of smoke .

Hence it decreases local wavelet

energy in the scene

Researc

h a

nd

Imple

me

nta

tion

of 61


we calculate difference of LWE of

background frame and Current frame

to get this feature

Researc

h a

nd

Imple

me

nta

tion

of 61


Backgroun

d

Current

Frame

Researc

h a

nd

Imple

me

nta

tion

of 61

Results

0 20 40 60 80 100 120 140 160 180 200-5

0

5

10

15

20x 10

-3 Change is Wavelet Energy (eb-e)

Video Frames

Change in W

avelle

t E

nerg

y

0 20 40 60 80 100 120 140 160 180 200-0.3

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05Change is Wavelet Energy (eb-e)

Video Frames

Change in W

avelle

t E

nerg

y

Fire

Sm

oke

Vid

eo

Hum

an M

ovem

ent

Vid

eo

Researc

h a

nd

Imple

me

nta

tion

of 61

Source Stability

Source of fire smoke always remain

near about at same location while in

case of a human movement complete

foreground object moves and there is

not a single emerging (source) path.

Researc

h a

nd

Imple

me

nta

tion

of 61

Source Stability

Researc

h a

nd

Imple

me

nta

tion

of 61

Results

Researc

h a

nd

Imple

me

nta

tion

of 61

Results Comparison

0 100 200 300 400 500 600 700 8000

2000

4000

6000

8000

10000

12000

14000Source Stability VS Frame No

Frame No

Sou

rce

Sta

bili

ty

0 50 100 150 200 250 300 350 400 4500

500

1000

1500

2000


Frame No

Sou

rce

Sta

bilit

y

0 50 100 150 200 250 300 3500

100

200

300

400

500


Frame No

Sou

rce

Sta

bili

ty

0 100 200 300 400 500 600 700-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8


Frame No

Sou

rce

Sta

bili

ty

Fire

Sm

oke

Vid

eo

Hum

an M

ovem

ent

Vid

eo

Researc

h a

nd

Imple

me

nta

tion

of 61

BP Neural Networks

BP Neural Networks are trained using

logsig training function with several

smoke and non-smoke videos.

Later this trained Network is used for

real time smoke detection.

MATLAB is use to train the NN

Pre-

Processing

Moving

Target

Detection

Feature

ExtractionSmoke

Detection

Researc

h a

nd

Imple

me

nta

tion

of 61

BP Neural Networks

1. Growth Rate

2. Disorder

3. Number of Segments

4. Frequent Flickering

5. Local Wavelet Energy

6. Source Stability

0.65 & & 1 Sm oke

0.25 & & 0.64 D anger

0 & & 0.24 N orm al

out

Researc

h a

nd

Imple

me

nta

tion

of 61

Features Weights for NN

Results

Growth Rate

Disorder

Number of Segments

Frequent Flickering


Source Stability

Smoke Starts Smoke

Spreads

Researc

h a

nd

Imple

me

nta

tion

of 61

Tra

inin

g o

f the N

N

Construction of Network

Samples Completed

Frame Extraction

Video Samples

(8 Videos)


Feature Extraction

Save Feature Vector

Training of Neural Networks

of 61

Training of the NN

Researc

h a

nd

Imple

me

nta

tion

of 61

Use o

f Tra

ined N

N

of 61

RESULTS

of 61

Results of Smoke Detection

Results

of 61

NN Results (Smoke Video)

Results

of 61

NN Results (non-Smoke

Video)

Results

of 61

NN Results

Results

of 61

DEMOS

of 61

Demos

Smoke Video 1

Smoke Video 2

Smoke Video 3

Smoke Video 4

Human Video 1

Human Video 2

Human Video 3

Human Video 4

Videos\STestRestep05.avi







Videos\MTestRestep01.avi









of 61

QUESTIONS / ANSWERS

THANKS EVERYONE FOR

YOUR TIME

research and implementation of smoke detection in video streams [email protected]

Business