matlab 및simulink를이용한운전자 지원시스템개발 · code generation. 14 ... access...

1© 2015 The MathWorks, Inc.

MATLAB 및 Simulink를이용한운전자지원시스템개발

김종헌차장

Senior Application Engineer

MathWorks Korea

2

Example : Sensor Fusion with Monocular Vision & Radar

Configuration

– Monocular Vision installed on Wind Shield

– Radar installed in Front Grill

High Level Sensor Fusion

2 Different Scenario

– Closest In-Path Vehicle Selection

– Stationary Vehicle Detection

3

Why Sensor Fusion?

– To complement weakness of individual sensor

– More Coverage

Radar Camera

Accurate Range Poor

Measureable Directly Range Rate Estimated

Poor Angle Resolution Relatively Good

Not Available Classification Available

Sensor Fusion

4

Issues?

Max Doppler

RCS

InterferenceNoise figure

Power@50

SGM

KLT Tracking

HOG

TTCLQR/LQG

AUTOSAR

Communication Risk of Vehicle Test

Huh!!

5

How do we bridge the gap between engineers

who speak different languages and reduce risk?

Max Doppler

RCS

InterferenceNoise figure

Power@50

SGM

KLT Tracking

HOG

TTCLQR/LQG

AUTOSAR

Communication Risk of Vehicle Test

6

Solution: Model based Multi-domain Modeling and System Level

Simulation

“Here’s the

verification

model”

“Here are

the specs”

• Model serves as the executable specification and test environment

• Multiple engineering teams can collaborate more effectively

7

Example : Sensor Fusion with Monocular Vision & Radar

Vision

– Lane Detection

– Vehicle Detection

– Extended Kalman Filter based Tracking

Radar

– FMCW Radar Modeling

– Extended Kalman Filter based Tracking

Sensor Fusion

– Data Association

– Data Fusion

8

Abstract

Vision

– Lane Detection


– Etc.

Radar


Tracking

– Extend Kalman Filter based Tracking

Sensor Fusion

– Integration

9

Captured Image

ROI (Region Of Interest)

Cropped Image


Curve FittingSelecting Candidate PtsEdge Mapping to World

Lane Detection

10

Captured Image


Cropped Image


Curve FittingSelecting Candidate PtsEdge Mapping to World

Lane Detection

Image Processing Function

MATLAB Built-In Function

CVST Function

Easy Matrix Calculation

Easy to Debug Scripting

Language

Pixel level, 2D, 3D Visualization

Code Generation

14

Lane Detection

15

Abstract

Vision

– Lane Detection


– Etc.

Radar


Tracking


Sensor Fusion

– Integration

16

Cascade Detector

Applications in Automotive of Cascade Detector

Vehicle detection

Pedestrian detection

Traffic Sign Recognition

Difference between applications is “Feature”.

17

4 Easy Steps to Create a Car Detector

Collect Car Image Data

(Training Images)

Label Objects of Interest

Train Object Detector

Test Detector Performance

18

Step 1: Collect Training Image Data

Collect many images that represent object you want to

detect

Ensure that images represent different conditions

– Lighting

– Different variations of object – size, color , model etc.

19

Step 2: Label Training Images

Select and assign regions of interest

Images from: www.emt.tugraz.at/~pinz/data/GRAZ_02/

20

Step 3: Train Object Detector

Single line of ML

code

21

Step 4: Test Detector Performance

22

Comparison Data from Vehicle Dynamics S/W

and Result of Cascade Detector

23

Further Information about Cascade Object

Detector

See documentation for detail explanation http://www.mathworks.com/help/vision/ug/train-a-

cascade-object-detector.html .

http://www.mathworks.com/help/vision/ug/train-a-cascade-object-detector.html

24

Abstract

Vision

– Lane Detection


– Etc.

Radar


Tracking


Sensor Fusion

– Integration

26

Camera Calibration App.

Original image Undistorted image


MATLAB commands for camera calibration are also available.

27


𝐾 =

𝑓𝑥 0 0𝑠 𝑓𝑦 0

𝑐𝑥 𝑐𝑦 1

𝑐𝑥, 𝑐𝑦 - camera center in pixels

(𝑓𝑥, 𝑓𝑦) - focal length in pixels per world unit

– 𝑓𝑥 =𝑓

𝑝𝑥, 𝑓𝑦 =

𝑓

𝑝𝑦

𝑠 – sensor axes skew coefficient

– 𝑠 = 𝑓𝑦 tan𝛼

Intrinsic Matrix

Radial Distortion

• 𝑥𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 = 𝑥𝑟𝑎𝑑𝑖𝑎𝑙 + 2𝑝1𝑥𝑦 + 𝑝2 𝑟2 + 2𝑥2

• 𝑦𝑡𝑎𝑛𝑔𝑒𝑛𝑡𝑖𝑎𝑙 = 𝑦𝑟𝑎𝑑𝑖𝑎𝑙 + 𝑝1 𝑟2 + 2𝑦2 + 2𝑝2𝑥𝑦

𝑥𝑟𝑎𝑑𝑖𝑎𝑙 = 1 + 𝑘1𝑟2 + 𝑘2𝑟

4 + 𝑘3𝑟6 𝑥

𝑦𝑟𝑎𝑑𝑖𝑎𝑙 = 1 + 𝑘1𝑟2 + 𝑘2𝑟

4 + 𝑘3𝑟6 𝑦

𝑥 and 𝑦 are in normalized image coordinates

– in world units relative to the optical center

Tangential Distortion

28

Image Processing

Image Enhancement

– Deblurring

– Noise Removal

Image Segmentation

– Active Contours

– Watershed

29

Feature Detection, Extraction and Matching

Corner Template SURF MSEREdge

Image MatchingFeature Point Tracking

Video Stabilization

30

Stereo Vision (2014a)

Stereo calibration

Semi-global disparity matching

– Better results than block matching

3D scene reconstruction from

disparity

New demos available

– Estimate real distance to objects in

video

– 3D scene reconstruction

– Pedestrian detection (2014b)

31

Stixel based Object Detection

Available to detect objects which are not easy using

other sensors

– Stationary object, overlapped object, kid….

* Stereo and City, Franke Uwe, Daimler Research Center

32

Disparity Image

Stixel Representation

3D Reconstruction

Bird View

KLT Point Tracking

Stereo Vision Demo

Computer Vision System Toolbox

– Stereo camera calibration

– SGM based Stereo matching

– 3D reconstruction using camera

calibration data

– Kanade-Lucas-Tomasi point tracking

– 2D/3D plotting

33

Point Cloud Processing (2015a)

pointCloud object

– Fast search using kdtree

– Tree is constructed implicitly when a

search is called first time

Downsampling

– Two method : ‘gridAverage’, ’random’

Visualization

Rigid registration

– ‘Iterative Closest Point’ Algorithm

34

OCR – Optical Character Recognition (2014a)

Highly requested in Frontlines

Support for English, Japanese

Users can download additional

language support

Shipping demo with text detection

and OCR workflow

35

Abstract

Vision

– Lane Detection


– Etc.

Radar


Tracking


Sensor Fusion

– Integration

36

Modeling FMCW Radar for Automotive

The received signal is a delayed copy of the

transmitted signal

Ultra-large signal bandwidth (>100MHz)

Ultra high frequencies (>77GHz)

Car+Radar Target

37

FMCW Radar: How Does it Work?

PA

LNA

Target =

Delay

DSP

RF

time

delay

beat

frequency

fre

qu

en

cy

38

What Behavior Can Be Modeled?

PA

LNA

DSP

Determination of waveforms

(range, resolution)Algorithms for data analysis

Channel modeling

(interference, noise)

Modeling of antenna arrays

(no of elements, position)

Modeling of RF Impairments

(noise, non-linearity, frequency dependency)

39

Which MathWorks Tools Can Help?

PA

LNA

DSP

Phased Array System Toolbox

Signal Processing Toolbox

Instrument Control Toolbox


DSP System Toolbox

Communications System Toolbox Phased Array System Toolbox

SimRF

40

Waveform

GeneratorTransmitter

Transmit

Array

Signal

ProcessingReceiver

Receive

Array

Environment,

Targets, and

Interference


Waveforms

Pulse, LFM, FMCW, etc.

Tx Antenna Arrays

ULA, URA, etc.

Rx Antenna Arrays

ULA, URA, etc.

Beamforming, Matched

Filtering, Detection, CFAR,

STAP, etc.

Transmitter

Monostatic and Bistatic

Receiver


Environment effects,

impairments, interference

Algorithms

Simulation Tools

41

Waveform

GeneratorTransmitter

Transmit

Array

Signal

ProcessingReceiver

Receive

Array

Environment,

Targets, and

Interference


Waveforms

Pulse, LFM, FMCW, etc.

Tx Antenna Arrays

ULA, URA, etc.

Rx Antenna Arrays

ULA, URA, etc.

Beamforming, Matched

Filtering, Detection, CFAR,

STAP, etc.

Transmitter


Receiver


Environment effects,

impairments, interference

Algorithms

Simulation Tools

Design & Analyze

Tools

43

Radar Simulation

44

Abstract

Vision

– Lane Detection


– Etc.

Radar


Tracking


Sensor Fusion

– Integration

45

Moving From Detections to Tracking

Detect moving objects

Estimate motion of each object

Assign detections to tracks using data association

Innovate track data from detected objects

46

Kalman Filter

Kalman Filter (Linear)

Predict

a priori state estimate

a priori estimate covariance

Update

Innovation residual

Innovation covariance

Kalman gain

a posteriori state estimate

a posteriori estimate covariance

From Wikipedia,

47

Kalman Filter


Predict



Update

Innovation residual


Kalman gain



Function in MATLAB (Computer Vision System Toolbox)

– Define :

kalmanFilter = configureKalmanFilter(MotionModel, InitialLocation,

InitialEstimateError, MotionNoise, MeasurementNoise)

– Prediction :

pred_location = kalmanFilter.predict()

– Correction :

corrected_location = kalmanFilter.correct(observed_location)

48

Kalman Filter


Predict



Update

Innovation residual


Kalman gain



Blocks in MATLAB (DSP System Toolbox)

49

Extended Kalman Filter

Extended Kalman Filter

Prediction

Innovation

Prediction

Innovation

Jacobian Matrix !!!

50

How to Make Jacobian Matrix

From Wikipedia,

51

Solution

MuPAD in Symbolic Math Toolbox

Define X vector

Calculate 𝜕𝒇

𝜕𝒙

Define f vector

52

Solution

MuPAD in Symbolic Math Toolbox

53

Data Association for Tracking Multiple Objects

James Munkres's variant of the Hungarian assignment

algorithm

– [assignments,unassignedTracks,unassignedDetections] =

assignDetectionsToTracks( costMatrix,costOfNonAssignment)

– Example of costMatrix

costMatrix = zeros(nTracks,nDetect);

for i = 1:nTracks

costMatrix(i, :) = distance(predictedTracks(i), Detections);

endPredicted

location

Detection

54

Abstract

Vision

– Lane Detection


– Etc.

Radar


Tracking


Sensor Fusion

– Integration

56

Integration in Simulink

Input : Image &

Target Info.

Radar Module

. Radar

Modeling

. Detection

. Tracking

Fusion Module

. Data Association

. Data Fusion, TTC

. In-Path Vehicle

Visualization

Camera Module

. Lane Detection

. Vehicle

Detection

. Tracking

57

• Use a single file for multiple

engineers

• Chaotic

• Keeping track of file dependencies

How Can I Reduce Inefficiency

when Integrating Models?

What’s the problem which this

model has in terms of

“Integration”?

58

Partitioning a Model using Model Blocks

Model Reference used to

partition top-level model into

components

Signal Buses & Bus Objects

used to define the interfaces

between components

One file-per-component fits well

with Revision Control packages

Enabler for parallel development

of components

Parent Model

OLD MODEL

Child Model

59

One model Variation Management

Variant Subsystems

61

• Use a single file for multiple

engineers

• Chaotic

• Keeping track of file dependencies

How Can I Reduce Inefficiency

when Integrating Models?

What’s the problem which this

model has in terms of

“Integration”?

62

Simulink Projects

Search, manage, and share

project-related files

Conduct peer review of

changes using Simulink XML

comparison tool

Access source control

functionality for team

collaboration

Manage project-related files

efficiently within Simulink

63

Simulation Result

* : Vision Target

o : Radar Target

x : Fusion Target

Yaw Rate Based

Lane Based

64

Simulation Result

Stationary Vehicle

No Radar Target

65

Simulation Result

66

Conclusion

MATLAB/Simulink and etc. provide an environment to

develop algorithm, modeling environment and integrate

algorithms and various types of sensor models.

– For Vision or Lidar Sensor :

Image Acquisition Toolbox

Image Processing Toolbox

Computer Vision System Toolbox

– For Radar Sensor


SimRF

Signal Processing Toolbox

Simulink is a good platform for team collaboration

– Simulink Project

67

More Information

Computer Vision

– Computer Vision Workflow in MATLAB/Simulink

http://www.matlabexpo.com/kr/2015/sessions/t1-s2.html

– Object Detection and Tracking

http://in.mathworks.com/videos/computer-vision-with-matlab-for-object-detection-and-

tracking-81866.html?s_tid=srchtitle

– Face Recognition with MATLAB

http://in.mathworks.com/videos/face-recognition-with-matlab-98076.html?s_tid=srchtitle

Radar Design

– Modeling and Simulating Radar Systems Using MATLAB and Simulink

http://in.mathworks.com/videos/modeling-and-simulating-radar-systems-using-matlab-and-

simulink-81966.html?s_tid=srchtitle

– Design and Verify RF Transceivers for Radar Systems

http://in.mathworks.com/videos/design-and-verify-rf-transceivers-for-radar-systems-

81990.html?s_tid=srchtitle

Users Stories

– Traffic Recognition System from Continental

– Radar Sensor Alignment Algorithm from Delphi

http://www.matlabexpo.com/kr/2015/sessions/t1-s2.html

http://in.mathworks.com/videos/computer-vision-with-matlab-for-object-detection-and-tracking-81866.html?s_tid=srchtitle

http://in.mathworks.com/videos/face-recognition-with-matlab-98076.html?s_tid=srchtitle

http://in.mathworks.com/videos/modeling-and-simulating-radar-systems-using-matlab-and-simulink-81966.html?s_tid=srchtitle

http://in.mathworks.com/videos/design-and-verify-rf-transceivers-for-radar-systems-81990.html?s_tid=srchtitle

matlab 및simulink를이용한운전자 지원시스템개발 · code generation. 14 ... access...

Documents