camellia general

Wido Kruijtzer / Philips ResearchNovember 2003 CamelliaIST-2001-34410

Key Action IV.1.1

CAMELLIAGeneral overviewNovember 2003

Camellia ConsortiumContact:W.M. (Wido) KruijtzerPhilips ResearchProf. Holstlaan 4 (WDC31)Wido.Kruijtzer@philips.comTel: +31 40 2742025

CamelliaWido Kruijtzer / Philips ResearchNovember 2003

IST-2001-34410Key Action IV.1.1

Contents

• Smart Cameras• Motivation• Camellia project

– Applications• Algorithms

– Co-processor requirements– Co-processor Architecture

• Prototyping• Conclusions

IEEE Computer September 2002

Smart Camera application areas

Consumer

Significantly growing demand for SI applicationsin several domains

Automotive Mobile

Surveillance

Ecole des Mines - Paris

University of Las Palmas

University of Hannover

Camellia partners

Research – EindhovenPS-CoC-VC – HamburgPS-NxMM – Nijmegen

Start: April 1, 2002Duration: 33 monthsEffort: 28 pyIST-34410

ARM 9xxCPU

embed.RAM &(boot)ROM

CCIR /CameraFrontend

DataIF

Video Compression IP

VideoInput

SyncIF

DataIF

SyncIF

DataIF

SyncIF

DataIF

MemoryController(Flash & DRAM

DataIF

ext.Flash

Synchronization Infrastructure

Peripherals*

DataIF

SyncIF

ext.SDRAM

Camellia SmartImaging Coprocessors

I/O Interface

DataIF

off-chipcommunication

* - Timers, Watchdog, Interrupt Controller etc.

I/D Cache

DataIF

SyncIF

DataIF

SyncIF

DataIF

SyncIF

DataIF

SyncIF

MotionEstimator

TextureCodec

SmartImagingCopro 1

SmartImagingCopro 2

BistreamGenerator

Data Communication Interconnect

DataIF

SyncIF

DataIF

SyncIF

SmartImagingCopro 1

SmartImagingCopro 2

HW assist

Low-Cost and Low-Power Video Processor for Automotive and Mobile Applications with Intelligent Imaging Capabilities

Application selection

• Description of 7 automotive and 10 mobile applications– 1 paragraph general explanation– Application quotation based on:

• Customer interest• Compatibility

– With compression core (mainly motion estimation)

• Feasibility– Can we design it within the Camellia timeframe

• Benefits– How much value creation (Euros)

Applications selected for Camellia

Automotive• Low speed obstacle

detection (LSOD)• Pedestrian detection

Mobile• Image stabilization

• Face tracking

Reference:Camellia-D-2.1 “Applications”; November 2002

Application specifications

• Specification of each selected application– Functional requirements– Input/Output– Use cases

• E.g. face tracking:– Number of heads,– head coverage, size and orientation, – skin color, hairstyle, wearables, etc.

– Scenarios (combination of use cases)• Video sequences to reflect specific scenario

Algorithms

• General framework Particle Filtering*

– The theory behind it :• Bayesian approach of estimation“ A tutorial on Particle Filter for On-line Non-linear/Non-Gaussian

Bayesian Tracking”Sanjeev Arulampalam, Simon Maskell, Neil Gordon and Tim Clapp.

*Except for Image stabilization• motion estimation + motion field filtering

Particle FilteringGeneral architecture

ParticleFiltering

Output stage

ImageProcessing 1

Particles updating using likelihood functions

Feedback to image processing

ImageProcessing 2

ImageProcessing 3

Particle Filtering: Benefits

• Same architecture for several applications

• Scalability :– State space of targets

• Number of particles per target• Number of particles used for feedback

• Usability on target architecture:– No floating point operations– Light computation load with 100 particles– Can be computed in parallel with the image processing

operations (2-stages pipeline)

Particle FilteringApplied to obstacle detection

Edgedetection

Symmetrydetection

Motiondetection

ParticleFiltering

Output stage

Lightsdetection

Shadowdetection

Low level algorithms

• Basic pixel processing– image filtering, morphological operations, region analysis etc.

• Camellia Image Processing Library– Open Source– uses IplImage structure to describe images

• good replacement to the popular but discontinued Intel IPL library• good complement to the OpenCV library

– http://camellia.sourceforge.net

Reference:Camellia-D-3.1 “Report on Computer Vision Algorithms”; V4; July 2003

Example shadow detection

Road Histogram

020406080

100120140160180

1 21 41 61 81 101 121 141 161 181 201 221 241

Pixel value

Series1

Utilizes:

Histogramming, Thresholding, Erosion, Labeling+Blob

Application demos

Automotive application: Low speed obstacle detection (LSOD)

Particle FilteringApplied to face tracking

Face Colordetection

Face ovaldetection

ParticleFiltering

Output stage

Face featuresdetection

Application demos

Mobile application: Face tracking

Particle FilteringApplied to pedestrian detection

Edgedetection

LegsDetection

Motiondetection

ParticleFiltering

Output stage

BodyDetection

Application demos

Automotive application: Pedestrian detection

Application demos

Mobile application: Image stabilization

Algorithms results

• C++ code of all 4 applications available

– References:• Camellia-D-3.2b “Report on Initial algorithms 1”;

July 2003– Describes Image stabilization and LSOD

• Camellia-D-3.3b “Report on Initial algorithms2”; October 2003– Describes Pedestrian detection and Face tracking

SI Core architecture (1)

• SI is expected to be an essential differentiator for future products

• Successful introduction requires– Low cost (small Si area) and low power (esp. for mobile)– Easy integration into SoCs (small system interference)

• Architecture must support different application domains– No dedicated solutions !– Flexibility and Scalability are key

• Trade-off between Flexibility and Efficiency– Programmability vs. Adaptation to Algorithms

SI Core architecture (2)

• Hierarchy/structure of applications comprises three layers– Bottom layer: Set of core algorithms (Toolbox)– Common for all applications

• Core algorithms are basic pixel processing, image filtering, morphological operations, region analysis etc.

– Mid layer: Application-specific medium level algorithms• Uses core algorithms

– Top layer: Application-specific high level code • Based on particle filtering, uses medium level algorithms

SI Coprocessor architecture (1)• Selected Approach: Hierarchical Control

– Level 1: Micro-Instruction Level– Level 2: Macro-Instruction Level

• Macro-instruction initiates the execution of a sequence of micro-instructions for an image segment

• Only macro-instruction level is visible to the system

– Dedicated Macro-Instructions for each Algorithm mapped onto Coprocessor

• Implementation of macro-instruction functionality at design-time• Selection of processed image region at run-time

– Benefit: Reduced System Interference while sustaining Flexibility for a Variety of Applications

Memory

Datapath

Control

Data I/O

RegFile DataRAM 1a

DataRAM 1b

DataRAM 2

MacroControl

UnitMicro

ControlUnit

MacroMem

Data Bus

Ctrl Bus

SI Coprocessor

Arithmetic 1

Arithmetic 2

Accu Regs

SI Coprocessor Architecture (2)

Arithmetic Unit 2 (1 Arithmetic Stage)Support for processing of 4 pixels in parallel

Arithmetic Unit 1 (3 Arithmetic Stages)Support for processing of 4 pixels in parallel

Accu Regs

ADD/SUB SHIFT/CLIPControlWord (Arith2)

to local memory

Operands from local memory

ADD/SUB MIN/MAX MUL DIV SHIFT

INV/ABS

ADD/SUB

ControlWord (Arith1)

Coprocessor Datapath

SI Coprocessor Architecture

• Implementation of a synthesizable SystemC model of the SI coprocessor

• Performance analysis– 100 MHz target clock frequency, 32-bit arithmetic unit

Real-time processing of up to SDTV resolution (720x576 pixel, 25 Hz frame rate)

• Silicon area of SI Coprocessor (first results)– Logic synthesis for a 0.12 micron CMOS process

Coprocessor Area: around 1 mm2

Reference:Camellia-D-5.1a “Coprocessor specification 1”; October 2003

CoproSCModel

• SystemC-based reference implementation of SI-Core– Implemented in SystemC/C++ with graphical front end

• GUI uses WxWindows

– Used for the following tasks:• Definition of Architecture, micro- and macroinstructions• Performance Analysis• Architectural Refinement

– Thought as „Executable Specification“• Cycle-True Model of SI core• Reference for Implementation of architecture• Supplement to „Written Specification”

CoproSCModel

MemoryModel

CPUModel

SI-CoreModelSI-

Handshake

Data, Address

Settings,Macro Ins.

Handshake

VideoIO

High-Level Ctrl

CoproSCModel Demo

Downscale

Symmetry Detection

SI Core System integration

SI Core

SoC SyncWrapper

SoC DataWrapper

SI Coprocessor

Simple MI IF

Physical TTL

SoC Sync Infrastructure

SoC Data Communication Interconnect

Camellia System Simulation Environment (CSSE)

• System simulation environment– Implemented in SystemC/C++ with graphical front end

• Uses SystemC Master/Slave library• GUI uses WxWindows

– Investigation of selected system aspects• Obtain metrics of communication load and performance estimations• Make system and application optimizations

– Platform for mapping of applications• Check functional correctness

– Functional verification of SI Coprocessor

CSSE Architecture (TTL based)

CPU SMART IMAGING

CPU WRAPPER WRAPPER

MEMORY

Synchro bus arbiter

Data bus

Config bus

Ack / Req

MASTER

WRAPPER

Synchronization bus

Data bus arbiter

Coproc IF(TTL Layer)

CSSE Architecture (Logical view)

hlamla1

SIlla1

macro_si_out port

macro_si_in port

macro_sichannel

Video_out port

Video_in port

video channel

memory_outport

memory_inport

memory_outport

memory_inport

memory_outport

Frame Memory

hlamla1

hlahlamla1mla1

mla2mla2

mlanmlan

SIlla1

lla1lla1

lla2lla2

llanllan

macro_si_out port

macro_si_in port

macro_sichannel

Video_out port

Video_in port

video channel

memory_outport

memory_inport

memory_outport

memory_inport

memory_outport

Frame Memory Synchronous channels

Asynchronous ports and channels

Synchronous ports

CSSE demo

Camellia prototyping system

• Build 2 prototypes• Prototype used in 2 modes

– Functional• Uses video sequences

– Field test• Automotive

– Integration with frame-grabber and CAN-bus controller

• Mobile– Integration with CMOS cameras and mobile display

• Prototype based on Avalon

Main AVALON features

• PCI board prototyping system• Contains 2 Altera’s PLDs (up to 808 I/O)

– One APEX 20KE device with 1.5M gates– One Altera’s Excalibur with 1M gates + ARM922T hard core (200 Mhz)

• Processor subsystem connection to 128Mb DDR SDRAM at 266Mhz• 144 pin SODIMM interface on each PLD• User I/O

– 430 signals between PLDs (32 via switch matrix)– 181 signals to expansion connectors– 48 signals to 68 pins bracket connectors– 96 signals trough advanced streaming port PrEDICT

Prototyping Hardware

Altera: Philips Avalon

AVALON plug-in board

AVALON - Excalibur inside

Main results

• A set of basic SI algorithms• A set of specific applications for automotive and mobile

domain• A low-cost low-power core for efficient acceleration of SI

algorithms to support the applications• 2 demonstrators showing the effectiveness of the applications

using the Camellia system prototype

• Explored advanced design methodologies for improving TTM• Contribution to sustaining the competitive advantage of

Europe in the mobile industry• Accelerated introduction of smart cameras in European cars

camellia general

Technology

kamelien · 2019. 2. 25. · kamelien sortenliste 2018...

korelasi konsentrasi ekstrak teh hijau (camellia...

pemanfaatan ampas teh (camellia sinensis) sebagai pakan...

h2fa1 camellia nantes frans

camellia sinensis...

camellia villas

sd herbssdherbs.com/pdf/brochure.pdf · garcinia combogia...

quarteiras 02h00 extensão...

ii pengaruh pemberian seduhan teh hitam (camellia sinensis

camellia cml classic nd yag laser use manual spain

manajemen pemangkasan tanaman teh (camellia sinensis …

hubungan kebiasaan minum teh hitam camellia …

ir-perpustakaan universitas...

34 pengaruh penggunaan ampas teh (camellia sinensis

pengelolaan pembibitan pada tanaman teh (camellia …

estudo da ação inibidora do extrato de camellia sinensis

análisis de camellia síntesis (té verde)

pengendalian gulma tanaman teh (camellia sinensis l.) di

1)-59.pdfhibiscus mutabilis hibiscus syriacus camellia...

camellia villa ajman uptown