Speaker

941028

MPEGVLSINSC90-2213-E-327-010

IC Type

RF ICMemory ICPower ICMixing Mode ICMicro Processor & Interface chipAnalog ICDSP IC

DSP Chip=Algorithm +VLSI architecture +Chip Implementation

Inter-coding/Predictive-coding Intra-coding

I

P3

P1

P2

B0

B1

B0

B1

B0

B1

1

10

9

8

7

6

5

4

3

2

........

Forward Prediction

Bidirectional Prediction

B0

B1

I

11

12

13

Blockinput

DCT

Quantization

Zig-ZagandRun-Length

VLC

Buffer

InverseDCT

InverseQuantization

MotionEstimation

FrameMemory(coded)

CodingControl

Intra-coding

Inter-coding

Intra-coding

Inter-coding

+

+

+

-

To Quantization

To Mode Switch

To Motion Estimation

Buffer

VLCDecoder

Inverse Quantization

InverseDCT

Frame Memory

Motion Compensation

Loop Filter

Mux

0

Motion vector

Inter/intra mode

Input data

Decoded data

Inverse Zig-ZagInverse Run-length

intra

inter

Video sequence

sequence header

GOP

GOP

GOP

K

Sequence end code

GOP header

Picture

Picture

Picture

K

Picture

Picture

Picture header

Slice

Slice

Slice

K

Slice

Slice

Slice header

MB

MB

MB

K

MB

MB

MB header

block

block

block

K

block

block

Transform Coefficients(TCOEFF)

K

End of block(EOB)

Sequence Layer

GOP Layer

Picture Layer

Slice Layer or GOB Layer

MB Layer

block Layer

VLCVLCIBBPBBPBBPBBI (Forward error correction) (ARQ)

>Bilinear

dT

dL

dB

dR

bT(0,k)

b(i,k)

bB(N+1,k)

bL(i,0)

bR(i,N+1)

N=16

>zero motion vector

t

t-1

>above motion vector

t

t-1

MV=(x,y)

MV=(x,y)

>average motion vector for I-frame

tIntra-frame

t-1P-frame

MV=(x,y)

MV=(TLx,TLy)

MV=(BLx,BLy)

>average motion vector for P-frame

tP-frame

t-1P-frame

MV=(x,y)

MV=(TLx,TLy)

MV=(BLx,BLy)

>median motion vector of P-frame

tP-frame

t-1P-frame

MV=median{candidate MB}

MV=(x,y)

Corrupted MB

Candidate MB

>median motion vector of I-frame

Corrupted MB

tIntra-frame

t-1P-frame

MV=median{candidate MB}

MV=(x,y)

Candidate MB

>Boundary Matching Algorithm(BMA)

:Lam W.M., Reibman A.R., Liu B. , Recovery of lost or erroneously received motion vectors , 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing, 1993. ICASSP-93., Vol. 5, p.p. 417-420, 1993.

Region A

Region B

Region C

>Decoder Motion-Vector Estimation (DMVE) Algorithm

:Jian Zhang , John F. Arnold and Michale R. Frater , A Cell-Loss Concealment Technique for MPEG-2 Coded Video IEEE Trans. on Circuits and Systems for Video Technology, Vol. 10, No. 4, p.p. 659-665, June 2000.

Lost macroblock

Decoded pixels fromcurrent frame

Median Vector for P or I frame for P frameP-frameI-frameShih-Chang Hsia , Shyi-Chyi Cheng and Shih Wen Chou, Efficient adaptive error concealment technique for video decoding system, IEEE Trans. Multimedia. IEEE Trans. Multimedia, pp.860-868, Oct. 2005.

>

t-1

t

V

Corrupted MB

VqLW

t-1

t

Vq

Corrupted block

V

MV=(x,y)

t-1

V

V

Vq

t-2

MV=(x,y)

P-frame at time t

P-frame at time t-1

Corrupted MB

LTDlLTDtlLTDtrLTDrLTDbrLTDbl

To decide where is the object?

V

p

p

p

p

p

p

j

j

j

Caculate TDtl BTDtBTDtrBTDblBTDbBTDbr

LTDmin

>,

BMB=Min. (MAD(Mxt)), Mxtfrom N to N

(2)

(1)

(3)

(4)

(5)

,BMB=Min. (MAD(Mxb)), Mxb from N to N

>P-frame

ft-1(Mv),IDCT(Fuv) >Block Deviation()

BDTL

BDT

BDTR

BDBL

BDB

BDBR

Corrupted MB

Correctly decoded MB

BDx:Block Deviation for xMB

coeff_P

>I-frame

D45,D135,V1AH

BV(Block Variance)

SIlost

MDlost(Motion Distance)

MF(Moving Feature)

coeff_I

I or P

pCMBBD

BDlost=BDlostmax

BDlost=BDlost

plBDlost

BDlost>BDlostmax

coeff_P

MDiff>MDiffmax

P-frameAVX

MDiff

H~Bz

pCMBAH

pCMBBV

plAHlost

plBVlost

yes

P

I

No

No

yes

AHlost=AHlost

BVlost>BVlostmax

BVlost=BVlostmax

AHlost=AHlostmax

AHlost>AHlostmax

BVlost=BVlost

plMDlost

MDlost>MDlostmax

MDlost=MDlostmax

MDlost=MDlost

MF

coeff_I

coeff_I

P-frameI-frameComplex Video

Peak Signal to Noise Ratio(PSNR)Io : original imageIr : result imageMSE : mean square errorH : height of imageW : width of image

,

,

,PSNR=40.7dB

bilinearPSNR=39.72dB

4CIF (704x576) GOBMB12~MB22 10(49) for P or I frame 1(271) for Complex Video

CIF1

CIF2

CIF3

CIF4

GOB1

GOB2

GOB3

GOB4

GOB5

GOB6

GOB7

GOB8

GOB9

GOB10

GOB11

GOB12

MB2

MB3

MB4

MB5

MB6

MB7

MB8

MB9

MB10

MB11

MB1

MB13

MB14

MB15

MB16

MB17

MB18

MB19

MB20

MB21

MB22

MB12

MB24

MB25

MB26

MB27

MB28

MB29

MB30

MB31

MB32

MB33

MB23

704

576

176

48

P-frameOur approach-1(median) Our approach-2LTD_threshold=6LVD_threshold=20BDlostmax=2048

PSNR=11.24dB Frame number=6

bilinearPSNR=26.6dB

DMVE,PSNR=24.24dB

our approach-1,PSNR=26.94dB

Frame number=6PSNR=15.69dB

bilinearPSNR=33.65dB

DMVE, PSNR=34.46dB

our approach-2 PSNR=35.98dB

Frame number=6PSNR=14.3dB

bilinearPSNR=29.62dB

DMVE, PSNR=26.85dB

our approach-1 PSNR=29.88dB

I-frameOur approach(median)MDiffmax=16384BVlostmax=512AHlostmax=2048MDlostmax=16C1=0.4C2=0.6

Frame number=12PSNR=12.37dB

bilinearPSNR=37.28dB

DMVE,PSNR=28.33dB

our approach,PSNR=36.78dB

Frame number=12PSNR=9.54dB

bilinearPSNR=21.33dB

DMVE,PSNR=32.97dB

our approach,PSNR=31.98dB

Frame number=12PSNR=15.49dB

bilinearPSNR=41.14dB

DMVE,PSNR=31.18dB

our approach,PSNR=39.69dB

Complex VideoOur approach-1P or I frame(median) Our approach-2P frameI frame(median)

Frame number=192(I-frame)PSNR=13.28dB

bilinearPSNR=25.57dB

DMVE,PSNR=20.28dB

our approach-1,PSNR=25.45dB

(median)bilinearBDlostmax=2048BVlostmax=512AHlostmax=2048MDlostmax=16C1=0.40625=(0.01101)bC2=0.59375=(0.10011)b

GOB1

GOB2

GOB3

KK

704

16

MB1

MB2

MB3

MB4

MB44

KK

AVX

~

Atkp

~

Recovery motion vector

Motion vector input

DCTY

H

Clk,reset,Frametype,errordet,gobstart,mbdatastart,mba,cbp

frame memoryJ

From frame memory

MUX

X

Spatial MVy Storage

Temporal MVy Storage

mba,cbp,mbdatastartgobdatastart,frametype

AHlost,BDlost,BVlost,MF,SIlost decision ,coeff_I and coeff_P computation

Head processing

VLC/RLC

IQ

IDCT

Buffer

Frame memory

Line buffer704x8

Line buffer16x8

Bilinear Interpolation

Adaptive Algorithm for I or P Frame,

latch

MUX

Error Detection

Moving Feature Computation(MDlost)

Spatial MVx Storage

Temporal MVx Storage

MUX

Block Deviation or Block Variance computation and storage

H component computation and storage

Zig-Zag

Line Bufferread/writecontrol

Frame memoryaccess control

Control signalgenerater

MUX

MV storage access control

Median Vector and MDlost computation Control

Buffer

Line buffer32x8

MB Boundry

Continue two GOBs error

frametype

ErrorCorrect

bitstream

errordet

(Moving Feature)

FF

MUX

D

Q

MUX

FF

D

Q

0

6

KK

KK

MV Buffer

MDlost

From MV storage

0

FF

D

Q

MUX

MUX

0

0

BD or BV or AH

MUX

FF

D

Q

j

dctY

To Storage

MUX

0

MB1

MB2

MB3

MB4

MB5

MB45

MB46

MB47

MB48

MB49

MB89

MB90

MB91

MB92

MB93

Corrupted MB

MB44

MB88

MB132

KK

KK

KK

MB47~I

MB47~AVXI

1.MB472.3.errorcountdown=42 (88-47+1=42)

4.errorcountdown=05.GOB,MB6.MB89cbp=607. countpix=255errorflag

1.MB922.3.Buffer4.Recmvx=Median(1320323)=2Recmvy=Median(4321656)=4

bilinear

MB2

MB3

MB4

MB5

MB46

MB47

MB48

MB49

MB90

MB91

MB92

MB93

Write in top line buffer

Read out from top line buffer and write in top buffer

Write in bottom buffer

bilinearI

top line buffer top buffer 1.MB42.MB3 bottom line line bufferWAwrite addressline buffer

3.MB915.line bufferMB3 bottom line 4. line bufferMB3bottom lineread address6.MB937.top bufferbilinear

bottom buffer 1.MB922.MB91top linebottom buffer3.MB934.MB(256 clocks)MB91top line

Bilinear1.MB932.16clocks3.

BD or BV1.MB32.3. MB3dctBV=1886

4.MB935.MB93BV(BVt)

AH 1.MB32.MB3AH3. MB3dctAH=174

4.MB935. MB93AH(AHt)

coeff_I , coeff_PMB93

MB5

MB2BV=1809AH=239

MB3BV=1886AH=174

MB4BV=1777AH=182

MB49

MB47BVT=1886BVB=0AHT=174AHB=0MDlost=5BVlost=512;AHlost=174

MB48

MB93

MB90BV=1892AH=247

MB91BV=0AH=0

MB92BV=0AH=0

MB93

BLPs->LBwrite Bottom Line Pixels into Line BufferLB->topBread from Line Buffer and write into top BufferRAThe Read address of top line bufferWAThe Wirte address of top line buffer

MB1 DCTX

MB2 DCTX

pMB2AHMBV

256 clocks

GOB1 start signal

256 clocks

MB44 DCTX

pMB44AHMBV

256 clocks

pMB1AHMBV

16 clocks

NMB1BLPs->LB (WA=0~15)

1 clocks

16 clocks

MB1 DCTX

NMB43BLPs->LB (WA=672~687)

GOB2 start signal

MB3 DCTX

RA=0~15

3 clocks

pMB1AHMBV

256 clocks

1 clocks

MB2 DCTX

pMB2AHMBV

256 clocks

NMB1BLPs->LB (WA=0~15)

16 clocks

errordet

257 clocks

RA=16~31

256 clocks

1 clocks

NMB44BLPs->LB (WA=688~703)

16 clocks

2 clocks

RA=0

BV=0AH=0

LB->topB

LB->topB

2 cks

257 clocks

1 cks

256 clocks

42 clocks

1 clocks

GOB3 start signal

1 clocks

WA=0

RA=0~15

LB->topB

2 clocks

MB92MB93MB46MB47

RA=0~15

MB1 DCTX

pMB1AHMBV

WA=0

LB->topB

255 clocks

254 clocks

256 clocks

MB2 DCTX

pMB2AHMBV

256 clocks

NMB1BLPs->LB (WA=0~15)

16 clocks

RA=16~31

2 cks

256 clocks

LB->topB

256 clocks

MB3 DCTX

pMB3AHMBV

256 clocks

NMB2BLPs->LB (WA=16~31)

16 clocks

RA=32~47

256 clocks

1 ck

LB->topB

256 clocks

1 ck

MB4 DCTX

pMB4AHMBV

256 clocks

NMB3BLPs->LB (WA=32~47)

16 clocks

RA=48~63

256 clocks

LB->topB

256 clocks

1 ck

2 cks

2 cks

Load MV

7 clocks

GOB2_MB3_output=adaptive_algorithm_output

Median vector

21 clocks

bilinear

MB5 DCTX

256 clocks

1 ck

bilinear

pMB5AHMBV

256 clocks

NMB4BLPs->LB (WA=48~63)

16 clocks

RA=64~79

256 clocks

LB->topB

256 clocks

256 clocks

1 ck

bilinear

1 ck

2 cks

2 cks

RA=80~95

256 clocks

LB->topB

256 clocks

1 ck

MB6 DCTX

pMB6AHMBV

256 clocks

1 ck

256 clocks

Load MV

7 clocks

GOB2_MB2_output=frame_memory_output

Median vector

21 clocks

Load MV

7 clocks

Median vector

21 clocks

GOB2_MB4_output=adaptive_algorithm_output

NMB5BLPs->LB (WA=64~79)

16 clocks

Cell_based

Cell_based RTL-HDL (Gate-Level) (Place & Route) CCadence Verilog-XLSynopsys Design AnalyzerRTL-HDL Cadence Verilog-XLCadence Silicon EnsembleCadence DraculaTimeMill/PowerMill

RTL Level Simulation , Gate Level Simulation 50MHzPost Simulation (TIMEMILL )TT Corner :253.3V 50MHz FF Corner 253.3V 50MHz 03.6V 50MHz SS Corner 253.3V 50MHz 1253.0V 50MHz FS Corner 253.3V 50MHz SF Corner 253.3V 50MHz

bilinearout[7:0]Memin[7:0] Frametype[1:0]

error_occure_nextgob

nextmb

recmvx[4:0]

recmvy[4:0]

EXTVDD

out[7:0]

preset

errordet

gobstart

mbdatastart

Frametypr[0:1]

bilinearout[7:0]

clk2

INTVSS

Memin[7:0]

mba[3:0]

mba[4:5]

dctin[11:0]

EXTVSS

cbp[2:5]

mvyin[4:0]

cbp[1:0]

mvxin[4:0]

INTVDD

rambottomin[7:0]

ramtopin[7:0]

100 pins99 pins

45~135I or P framePSNRDMVE0.3~0.4dB PSNR2dBVLSI(CIC)(S35-91F)