image denoising 成功大學 資訊工程系 陳 培 殷. outline embedded system what cse students...
Post on 15-Jan-2016
221 views
TRANSCRIPT
Image Denoising
成功大學 資訊工程系
陳 培 殷
Outline
Embedded System
What CSE Students Can Do in Embedded
System Design
Example -Image Denoising
Demo
PCB
PC: a general-purpose computing system
PC
Pentium
PCBEmbedded System: a special-purpose computing system
Most embedded systems are designed for
1. special purpose (customizing and non-programmable)
2. real time applications 3. stable applications
4. automatic applications
Embedded System (1/2)
uPUART
MPEG
ROM
RAM
Embedded System (2/2)
Traditional embedded systems use low-level processors only.
ARM PCI
MPEGUSBFLASH
ROM
RAM
DSPAMBA
Advanced embedded systems
Multi-core
Application Everywhere!
Applications
Information Appliances (IA):
1. Smart phone, VOIP
2. Digital TV, set-top box
3. PlayStation
4. PDA, mp3 player
5. Camera, DV
6. Air-conditioner, microwave oven, refrigerator,
vacuum cleaner, sensor network
7. Motorcycles
8. Car (abs, engine firing, air bag) >100 processors
9. … Ubiquitous computing (many computers for everyone)
Requirements
1. Friendly user interface
2. Short time-to-market
3. Real-time (Speed)
4. Cost
5. Power consummation/dissipation
(cooling strategy and battery life)
Synthesis
Applications
傳輸距離: 100 m功率消耗: 2 W功能:影像傳輸、 語音傳輸
螢 幕: 176 x 220 pixel 65535 色 1.8 吋 TFT
其他
傳輸距離: 100 m功率消耗: 2 W功能:影像傳輸、 語音傳輸
螢 幕: 176 x 220 pixel 65535 色 1.8 吋 TFT
其他
Specifications
Hardware:CPU,
RAM, I/O…
Hardware:CPU,
RAM, I/O…
Software:C,C++
Software:C,C++
System Design
always @(posedge clk) begin if (sel1) begin out=in1; else out=in2; endend
always @(posedge clk) begin if (sel1) begin out=in1; else out=in2; endend
Component Design
Layout
Placement & RoutingFabrication
Marketing
System Development Flow
Testing
What CSE students can do in
Embedded System Design?
My personal Views: 1. Hardware Design -Chip
2. Hardware Design -System
3. Related Tools Design
4. Firmware Design
5. Software Design
6. ESL Design
Design Flow
Specification
System Architecture
Hardware Design Software Design
System Integration
System Verification/Testing
Hardware/software partition is
very difficult !!! (cost time)
IC Industry in Taiwan
晶粒測試及切割
晶圓切割
設計
導線架
測試 封裝 製造 光罩
晶圓
邏輯設計 封 裝
化學品
成品測試光罩設計
長晶
ARM PCI
MPEGUSBFLASH
ROM
RAM
DSPAMBA
Hardware Design -- Chip (1/4)
ASIC
The basic design flow for digital cell-based ASIC
Describe circuits with hardware description language
(HDL 硬體描述語言 ) VHDL and Verilog
Synthesis ( 合成 ) the circuits ….
Need the concepts of digital logic/system
application specific integrated circuit (ASIC 晶片 ) such as USB, MPEG, ….
Full custom design vs. semi custom (cell-based) design
Hardware Design -- Chip (2/4)
Hardware Design -- Chip (3/4)
always @(…) if (a==b) if (c==1) d=f; else d=1; else d=0; a
b
c
f
d
fca
b
d
Translate into Boolean Representation
Optimize + Map
HDL Source
Target Technology
Synthesis = Translation+Optimization+Mapping
Process of logic synthesis
Software Engineers play a very important role for EDA tools development !!!
Hardware Design -- Chip (4/4)
FPGA or CPLD
Real ASIC chip
less flexible, long design cycle, higher speed,
larger-scale production to reduce price
more flexible, shorter design cycle, lower speed,
lower utilization, suitable for smaller production
Standard cellStandard cell
PLDPLD
Fab (TSMC, UMC, ..)
Two implementations :
Xilinx, Altera
Hardware Design -- System
ARM PCI
MPEGUSBFLASH
ROM
RAM
DSPAMBA
ASIC
Input devices: keyboard, touch screen, switch, button, ..
Output devices: monitor, LCD, LED, …
Extended devices: compact flash card (CF), PCMCIA, SD
(for storage, wireless communication, I/O)
Power system:
Transmission Interface: PCI, USB, IEEE 1394, UART, bluetooth…
Bus: AMBA (Advanced Microcontroller Bus Architecture)
Input devices
Output devices
Related Tools Design
晶粒測試及切割
晶圓切割
設計
導線架
測試 封裝 製造 光罩
晶圓
邏輯設計 封 裝
化學品
成品測試光罩設計
長晶
Firmware Design
ARM PCI
MPEGUSBFLASH
ROM
RAM
DSPAMBA
ASIC
Devices drivers for I/O devices, extended devices, transmission interface
Assembly codes and C codes for some dedicated CPUs (ARM, 8086,..)
Architectures and instruction sets of different CPUs, DMA,…
Input devices
Output devices
Software Design
ARM PCI
MPEGUSBFLASH
ROM
RAM
DSPAMBA
ASIC
Input devices
Output devices
Embedded OS: WinCE, Palm OS, uC/OS, Linux, JAVA
Real time OS (time) as small as possible (memory)
Distributed embedded system (+ fault tolerance)
Application Software:
wireless communication, network, multimedia,
health, convenience, Web, ….
Porting a customized embedded system to different
machines is very difficult (need large modification)
FutureChip: tens of millions of transistors or more (.35, .25, .18, .09)
Design shifts from ASIC/board to system
System on a Board(printed circuit board)
System on a chip
uP FPGA
MPEGASIC
ATMROM
ROM
SW SW
SWSWPCB
uP Core SRAM
ROM
ATMMPEG
ROM
FPGA
Glue Logic
A/D Block
PCB
SOCSystem-on-a chip is possible
(the whole system is
built in a single chip)
SOC is industry trend
multi-core
Example: Mobile Phone
Voice only; 2 processors 4 year product life cycle Short talk time
Yesterday
Voice, data, video, SMS <12 month product life cycle Lower power; longer talk time
Today
• 5~8 Processors
• Memory• Graphics• Bluetooth• GPS• Radio• WLAN
Single Chip
DSP
Radio
FlashMemory
Processor
Source: EI-SONICS
What’s ESL tools ?
Describe a SOC design in a more abstract way.
Provide virtual prototypes for HW and SW implementation.
Help system integration in early design phase.
Help to verify the result after job partition.
Claim that design and verification can be done in minutes.
Electronic System Level (ESL) Tools
Suitable for students in CSE
Hardware Software Co-Simulation
SoC Designer
Example: Image Denoising
Noise arises During image acquisition
CCD camera, light levels, sensor temperature During image transmission
Interference in the transmission channel
NoiseDetection
NoiseRemoval
Noise Suppression Process
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
Hardware Design – Chip !
Drawbacks of conventional algorithms Heavy computation loading Iterations
Large computation loading Additional memory space
Our goals: Cost-effective Good performance Suitable for VLSI implementation Real-time requirement
Motivation
Low-Complexity Algorithms
Embedded System
1. Power consumption
2. Computing time (real time vs. high quality)
VLSI Implementation
1. Circuit cost
2. Circuit speed
3. Power consumption
Low-Complexity algorithms are necessary for
embedded systems and VLSI implementations.
Issues for VLSI Implementation
Circuit Cost fixed point vs. floating point
less complexity, moderate quality vs. high complexity, high quality
+, - vs. *, / 2*x left shift x 5*x 4*x + x
Functional units sharing
Circuit Speed Pipeline architecture
Delay reduction
Power Consumption Sleep
Shut down partially
Cost-Effective Denoising Method (1/4)
1.Find Min & Max in the mask
3.Check Suspected Noise
4.Choose eligible & representativeedge directions
5.Find the minimum difference
6.Reconstruct the corrupted pixel
2.Utilize Min & Max info.to detect noise
Noise Detection Noise Removal
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
Check suspected noise
Case 1 Case 2 Case 3 Case 4
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
Cost-Effective Denoising Method (2/4)
Choose eligible & representative edge directions
Case 1 Case 2 Case 3 Case 4
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
Cost-Effective Denoising Method (3/4)
Find the minimum difference and reconstruct the noisy pixel
Case 1
1,1ˆ
jif jif ,1ˆ
1,1ˆ
jif
1, jif jif , 1, jif
1,1 jif jif ,1 1,1 jif
Cost-Effective Denoising Method (4/4)
Architecture of the Denoising Chip
Line Buffer 1
Line Buffer 2
ShiftRegister
Edge Detection Unit
Min & Max Computation
Unit
Noise Detection
Unit
Image Out
Image In
Output Selection Unit
Shift Register Bank
R0R1R2
R3R4R5
R6R7R8
Line Buffer
Line Buffer1st Raw2nd Raw
1st Raw 1st Raw2nd Raw
1st Raw
3rd Raw
DataflowMax & Min
Computation Unit
Noise Detection Unit
Edge Detection Unit Output Selection Unit
Stage 1
Stage 2
Stage 3
Stage 4
Pipeline Stages
Demo
Source ImageSending Image
Denoising Circuit
FPGA Board
Receiving Image
Result Image
Software Program
Hardware Platform
PC
PC
Hardware Software Co-Simulation/Verification
PSNP calculation