Slide 1

Image/Video Coding Techniques for IPTV Applications Wen-Jyi Hwang ( ) Department of Computer Science and Information Engineering, National Taiwan Normal University, Taipei, 117, Taiwan Slide 2 Goal of This Talk Overview of Image/Video Coding Techniques for IP TV Applications Image Coding: JPEG, JPEG2000 Video Coding: H.261, MPEG1, MPEG2, H.263, MPEG4, H.264 Slide 3 WHY Compression ? DSL ~200 kbps ~ 1,000 : 1 ~ 10,000 : 1 ITU-R 601 166 Mbps SDTV broadcasting ~2 Mbps ~ 100 : 1 Dial-up modem, wireless link ~ 20 kbps Slide 4 Image Coding Techniques JPEG (Joint Photographic Experts Group) DCT-Based Technique Published in 1992 JPEG2000 Wavelet-Based Technique Published in 2000 Slide 5 Principles of Image Coding Technique Color Transform DCT/WaveletQuantization Bitstream Formation Slide 6 Color Transform (JPEG) Process the data in blocks of 88 samples Convert RGB into Luminance (Y) and Chrominance (Cr and Cb). Use half resolution for Chrominance (because eye is more sensitive to Luminance) Slide 7 DCT (JPEG) Transform each block of 88 samples into 64 DCT coefficients energy tends to be concentrated into a few significant coefficients Slide 8 Quantization (JPEG) Divide each DCT coefficient by an integer, discard remainder Typically, a few non-zero coefficients are left. Slide 9 Quantization (JPEG) Slide 10 Wavelet Transform (JPEG2000) Slide 11 Slide 12 Quantization(JPEG2000) Slide 13 Slide 14 Slide 15 Slide 16 Quantization for Resolution- Scalable Transmission Slide 17 Quantization (JPEG2000) Slide 18 Layers 1+2Layers 1+2+3Layers 1+2+3+4 Slide 19 Quantization for SNR-Scalable Transmission Slide 20 JPEG2000-Based Layered Transmission Slide 21 JPEG2000-Based Region of Interest (ROI) Encoding Slide 22 Slide 23 Comparisons (JPEG and JPEG2000) JPEG (64:1) JPEG2000 (64:1) Slide 24 Video Compression-History It has been customary in the past to transmit successive complete images of the transmitted picture. [...] In accordance with this invention, this difficulty is avoided by transmitting only the difference between successive images of the object. Slide 25 Features of Moving Pictures Moving images contain significant temporal redundancy successive frames are very similar Slide 26 Motion Estimation and Compensation The amount of data to be coded can be reduced significantly if the previous frame is subtracted from the current frame. Slide 27 Block-Matching Results of block- matching The predicted current frame The reconstructed previous frameThe current frame Slide 28 Predicted Frame and Prediction Error Predicted Current Frame Displaced Frame Difference (DFD, ) Slide 29 I, B and P Frames A video sequences may contain three different types of frames: (a)Intra frames, (I-frames) (b)Predicted frames (P-frames), (c)Bi-directional frames (B-frames) Slide 30 Basic MPEG Standard Encoding order: I 0, P 3, B 1, B 2, P 6, B 4, B 5, I 9, B 7, B 8. Playback order: I 0, B 1, B 2, P 3, B 4, B 5, P 6, B 7, B 8, I 9. Slide 31 Basic Video Encoder Entropy Coding Deq./Inv. Transform Motion- Compensated Predictor Control Data 0 Intra/Inter Coder Control Decoder Motion Estimator Transform/ Quantizer - Video in DFD Motion Vector Encoded DFD Reconstructed DFD Reconstructed current frame Predicted current frame Slide 32 Basic Video Decoder Encoded bitstream in Motion vector Encoded DFD Predicted current frame Reconstructed DFD Reconstructed current frame Slide 33 Entropy Coding Deq./Inv. Transform Motion- Compensated Predictor Control Data Quant. Transf. coeffs Motion Data 0 Intra/Inter Coder Control Decoder Motion Estimator Transform/ Quantizer - Standards: H.261, MPEG-1, MPEG-2, H.263, MPEG-4, H.264/AVC Video in Slide 34 Entropy Coding Deq./Inv. Transform Motion- Compensated Predictor Control Data Quant. Transf. coeffs Motion Data 0 Intra/Inter Coder Control Decoder Motion Estimator Transform/ Quantizer - Standards: H.261, MPEG-1, MPEG-2, H.263, MPEG-4, H.264/AVC Video in -pixel accuracy Slide 35 Fractional Motion Estimation Slide 36 Entropy Coding Deq./Inv. Transform Motion- Compensated Predictor Control Data Quant. Transf. coeffs Motion Data 0 Intra/Inter Coder Control Decoder Motion Estimator Transform/ Quantizer - Standards: H.261, MPEG-1, MPEG-2, H.263, MPEG-4, H.264/AVC Video in Adaptive block sizes... Slide 37 Adaptive Block Size Slide 38 Entropy Coding Deq./Inv. Transform Motion- Compensated Predictor Control Data Quant. Transf. coeffs Motion Data 0 Intra/Inter Coder Control Decoder Motion Estimator Transform/ Quantizer - Standards: H.261, MPEG-1, MPEG-2, H.263, MPEG-4, H.264/AVC Video in Multiple Past Reference Frames Slide 39 Advantages of Multiple Reference Frames Slide 40 Entropy Coding Deq./Inv. Transform Motion- Compensated Predictor Control Data Quant. Transf. coeffs Motion Data 0 Intra/Inter Coder Control Decoder Motion Estimator Transform/ Quantizer - Standards: H.261, MPEG-1, MPEG-2, H.263, MPEG-4, H.264/AVC Video in Generalized B-Frames Slide 41 Entropy Coding Deq./Inv. Transform Motion- Compensated Predictor Control Data Quant. Transf. coeffs Motion Data 0 Intra/Inter Coder Control Decoder Motion Estimator Transform/ Quantizer - Standards: H.261, MPEG-1, MPEG-2, H.263, MPEG-4, H.264/AVC Video in De-Blocking Filtering Slide 42 without deblock filtering with deblock filtering Slide 43 Comparisons (H.264 and JPEG2000) JPEG2000 H.264 Slide 44 Comparisons (MPEG-4 and H.264) Slide 45 MPEG-4 H.264 1 Mbps Slide 46 Concluding Remarks Image/Video Coding Techniques such as JPEG2000 and H.264 are able to provide high fidelity while significantly reduce bandwidth for image/video transmission. These coding techniques therefore are effective for IPTV and/or DVB applications with limited bandwidth resources.