ELE 488 F06

ELE 488 Fall 2006Image Processing and Transmission

(11-16-06, 11-21-06)

JPEG block based transform coding Beyond basic JPEG Spatial correlation Subband decomposition & coding Wavelet transform Zero tree Successive Approximation Quantization

Digital Video

11/16

ELE 488 F06

Review of Up and Down Sampling (without filter)

• Down sampling

• Up sampling

+----+ x[n]-->| ↓ M |--> y[n]=x[nM], +----+

Y(ω) =M

1

1

0

M

r

X(M

r 2).

Example: M = 2 Y(ω) =2

1X(

2

) +

2

1X(

2

)

+----+ u[n]-->| ↑ L |--> v[nL]=u[n] +----+ y[Ln+k]=0,0<k<L V(ω) = U(ωL). Example: L = 2 V(ω) = U(2ω)

ELE 488 F06

0

X(ω)

2/2 0

X(ω/2)

2/2

X(ω/2-)Y(ω)

(A)

0

X(ω)

/22 0

V(ω)

/22

(C)(D)

(B)

Lowpass and Highpass Decimation

+----+ +----+ +------------+ -->| ↓2 |----->| ↑2 |----->| LPF or HPF |--> +----+ +----+ +------------+ x y v x

ELE 488 F06

Subband Decomposition + CodingEncoding / Decoding

H0 – lowpass, H1 – highpass

Y(z) = (1/2)[ H0(z)G0(z) + H1(z)G1(z) ]X(z)+ (1/2)[ H0(–z)G0(z) + H1(–z)G1(z) ]X(–z)

If G0(z) = H0(z) = H1(-z) = -G1(-z),

Y(z) = (1/2)[ H0(z)G0(z) + H1(z)G1(z) ] X(z), alias free

Specified by a single filter H0(z)

ELE 488 F06

Filter Banks

Analysis – decompositionSunthesis – combination

ELE 488 F06

Filterbank and Discrete Wavelet Transform (Mitra – section 14.5, 14.6)

More subbands

ELE 488 F06

ELE 488 F06

Specified by a single filter H1(z)

ELE 488 F06Total # of u samples same as # of x samples

Analysis (decomposition)

ELE 488 F06

Analysis (decomposition)

ELE 488 F06

From x[n] to {uk[n]}

Discrete Wavelet Transform (forward DWT)

Mother wavelet

Scaling function (level k)

Discrete Wavelet Transform (forward DWT)

ELE 488 F06

Synthesis (combining)

ELE 488 F06

Inverse Discrete Wavelet Transform (IDWT)

ELE 488 F06

Example – Haar Wavelet

H0(z) = H1(-z)

Other wavelets

ELE 488 F06

ELE 488 F06

Examples of 1-D Wavelet Transform

• Note: low frequency components similar From Matlab Wavelet Toolbox Documentation

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

ELE 488 F06

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)2-D Wavelet Transform via Separable Filters

From Usevitch (IEEE Sig.Proc. Mag. 9/01)

Note numbering of freq bands

ELE 488 F06

Embedded Zero-Tree Wavelet Coding (EZW)

• Exploits multi-resolution and self-similar nature of wavelet decomposition– Energy is compacted into a small

number of coeff.– Significant coeff

• w.r.t. a threshold• tend to cluster at the same

position in each frequency subband

• Two sets of info. to code: – Where are the significant

coefficients? (significance map)

– What values are the significant coefficients?

Usevitch (IEEE Sig.Proc. Mag. 9/01)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

ELE 488 F06

Key Concepts in EZW • Parent-children relation among coeff.

– A coeff at level k spatially correlates with 4 child coeff at level (k-1) of same orientation

– A lowest band coeff correlates with 3 coeff.

• Coding significance map via zero-tree– Encode only high energy coefficients

• Need to send location info. large overhead

– Encode “insignificance map” with zero-trees

• Successive approximation quantization– Send most-significant-bits first and

gradually refine coeff. value– “Embedded” nature of coded bit-stream

• Improve image quality by adding extra refining bits

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

Usevitch (IEEE Sig.Proc. Mag. 9/01)

ELE 488 F06

EZW Algorithm and Example

• Initial threshold ~ 2 ^ floor(log2 xmax)– Put all coeff. in dominant list

• Dominant Pass (“zig-zag” across bands)

– Assign symbol to each coeff. and entropy encode symbols

• ps – positive significance• ns – negative significance• iz – isolated zero• ztr – zero-tree root

– Significant coeff.• move to subordinate list• put zero in dominant list

• Subordinate Pass– Output one bit for subordinate list

• According to position in up/low half of quantization interval

• Repeat with half threshold– Until bit budget reached From Usevitch (IEEE

Sig.Proc. Mag. 9/01)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

ELE 488 F06

1st PassU

MC

P E

NE

E6

31

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

1)

xmax = 53, 32< 53 < 64, To = 32

Only 2 coef in [32,64)

(32+64)/2 = 48

48<53<64 1, 32<34<48 0

ELE 488 F06

After 1st PassU

MC

P E

NE

E6

31

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

1)

ELE 488 F06

2nd Pass

From Usevitch (IEEE Sig.Poc. Mag. 9/01)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

To=32, T1=16,

16< 22 < 32,

(16+32)/2 = 24

ELE 488 F06

Zoom in and compare sharp edges and smooth regions

ELE 488 F06

Beyond EZW

• Cons of EZW– Poor error resilience– Difficult for selective spatial decoding

• SPIHT (Set Partitioning in Hierarchal Trees)– Further improvement over EZW to remove redundancy

• EBCOT (Embedded Block Coding with Optimal Truncation)– Used in JPEG 2000– Address the shortcomings of EZW (random access, error

resilience, …)– Embedded wavelet coding in each block + bit-allocations

among blocks

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)

ELE 488 F06

JPEG 2000: A Wavelet-Based New Standard

• Targets and features

– Excellent low bit rate performance without sacrifice performance at higher bit rate

– Progressive decoding to allow from lossy to lossless

– Region-of-interest (ROI) coding

– Error resilience

• For details

– David Taubman: “High Performance Scalable Image Compression with EBCOT”, IEEE Trans. On Image Proc, July 2000.

– JPEG2000 Tutorial by Skrodras et al IEEE Sig. Proc Magazine Sept 01

– Links and tutorials: http://www.jpeg.org/JPEG2000.htm

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)

ELE 488 F06

Examples

JPEG2K vs.

JPEG

From Christopoulos (IEEE Trans. on CE 11/00)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

ELE 488 F06

ELE 488 F06

DCT vs. Wavelet

• 3dB improvement?– Wavelet compression claimed to have 3dB improvement

over DCT-based compression– Comparison done on JPEG Baseline

• Improvement not all due to transforms– Improvement comes mainly from better rate allocation,

advanced entropy coding, & smarter redundancy reduction via zero-tree

– DCT coder can be improved to decrease the gap

"A comparative study of DCT- and wavelet-based image coding" Z. Xiong, K. Ramchandran, M. Orchard, Y-Q. Zhang, IEEE Trans. on Circuits and Systems for Video Tech.,  Aug 1999, pp692-695.

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

ELE 488 F06

• Motion pictures started with an argument:– “Do They Or Don’t They?”– Muybridge (1877) - 12 cameras to take 12 pictures of

running horse

• Limitation of human vision system– persistence and fusion

Motion Picture Television Digital Video

ELE 488 F06

Motion Picture

• Perception of motion - persistence and fusion• 1877: Muybridge - 12 cameras to take 12 pictures of running horse • 1882: Marey - one camera to take 12 pictures per second• 1888: Dickson - motion picture camera

– Kinetograph (patent 1893, 50ft perforated film, 40 fr/sec, battery, 1000lb)

– Kenetoscope , single viewer projector (47 ft film, 25c for 5 shows) • 1895: Lumiere - projector/camera (hand-crank, 16 fr/sec, 20lb)• 1919: De Forest - optical sound on film• 1926 Warner B: “Don Juan” (John Barrymore, NY Philharmonic)• 1900 - : Film industry• 1940’s: Television• 1980’s: Digital Video

Recording event Re-live past moments Editing Creation of fictitious events / objects Manipulate perceived REALITY

ELE 488 F06

From Television to Digital Video

• Broadcast Television (analog)– movie at home - why invent new technology?– mass market– influence of movie on development

• Key Steps– convert pictures to electric signal– send electric signal – convert electric signal to picture

• Comparison with motion picture• High Definition Television - analog digital, compression

• Video telephone - analog predecessor• Video conference - travel cost, people cost• Cable (narrowcast), satellite, interactive, ...

ELE 488 F06

Key Steps in Broadcast Television

• Convert picture to electric signal – video camera– initially only at TV studios, cost not as important– recording media (tape), editing, …– special equipment to convert movie

• Send electric signal – follow radio broadcast, needs spectrum allocation

from FCC– VHF (Ch 2 – 13), UHF (Ch 14 & up)

• Convert electrical signal to picture– cathode ray tube offers economic solution– flat panel: LCD, LED, plasma panel– future

ELE 488 F06

Forming Picture on TV Tube (black-white)

How many lines?

ELE 488 F06

How Many TV Lines?

dot

dot

Cannot resolve if

distance > 2000 x separation

N = 500

ELE 488 F06

NTSC (National Television Systems Committee)

• 525 lines– 2 dots less than 1/2000 of distance from eye are not

separated (merge into one)– Assume view at distance 4 times the screen height. Dots

on screen less than (1/2000) x 4 x H = H/500 apart are not separated by eye. No need to have more than 500 lines

– NTSC set 525 lines (475 active)• Movies in 1940 has 4:3 aspect ratio (width to height)• 25 or more pictures per second to see continuous motion• 50 or more pictures per second to avoid flicker

– movies use 24 frames/sec, each shown twice• 30 frames/sec with 2:1 interlace (60 even-odd fields/sec)

ELE 488 F06

Television Tube

ELE 488 F06

2:1 Interlaced Scanning

even field odd field

ELE 488 F06

Bandwidth of Broadcast Television

• Without interlace (progressive scan), 60 frames/sec– 500 lines alternating black and white gives 250 full cycles– each horizontal line has 250 x 4/3 ~ 350 full cycles– 60 (frames/sec) x 500 (line) x 350 = 10,000,000 cycles/sec

or 10 MHz (video ONLY)

• With 2:1 interlace, 5 MHz for video

• FCC assigns 6 MHz per broadcast channel– real usable bandwidth is less, MUCH less– actual resolvable lines per vertical height ~250

• Color insertion - must compatible with B/W receiver– Change R-G-B to Y-Cb-Cr – Y is luminance (brightness), Cb and Cr are chrominances– B/W sets converts Y to picture, – Color sets converts Y-Cb-Cr to R-G-B, then display

ELE 488 F06

Digital Video

• What drives digital video?– Information technology:

• electronics, communication infrastructure, storage, functionality, …

– HDTV

• R-G-B component video– 640 x 480 (pixel) x 3 (color) x 8 (bits/color) x 30 = 221 Mb/sec

• Y-Cb-Cr with subsampled Cb and Cr– 640 x 480 (pixel) x 1.5 (color) x 8 (bits/color) x 30 = 110 Mb/sec

• Compression - MPEG (motion picture expert group)– MPEG-1: CD-ROM, 1.5Mb/sec, 1.2Mb/sec for video,

352x240 (CIF), progressive scan, motion compensation– MPEG-2: extension of MPEG-1, interlace, HD– MPEG-4: object/region based– H.2xx