xilinx standard templatesdi/aes router ird microphones camera intercomm sync pulse generator...

4K/8K高速影像介面及影像處理解決方案

May, 2017

© Copyright 2017 Xilinx.

Agenda o Video Market and Application

Overview

o Video Interfaces

o Video Processing

o Video Codec


Broadcast Chain (SDI)

Production Switcher

Multiviewer

Video/Audio Server

Camera

Playout Server

Secondary Distribution Encoder

SDI/AES Router

IRD

Microphones

Camera

Intercomm

Sync Pulse Generator

Satellite/Terrestrial/Cable Modulator

Primary Distribution Encoder

Contribution Encoder

Mini Converter

Video/Audio Processing

To Consumer

To Headend

To Regional Studio

SDI

ASI


Broadcast Chain (IP)

Ethernet SwitchesCOTS Servers

Multiviewer

Video/Audio Server

Camera

Playout Server

Secondary Distribution Encoder

IRD

Microphones

Camera

Intercomm

Sync Pulse Generator

Satellite/Terrestrial/Cable Modulator

Primary Distribution Encoder

Contribution Encoder

Mini Converter

Video/Audio Processing

To Consumer

To Headend

To Regional Studio

Cable/Terrestrial/Satellite

ASI

Internet/OTT

IP

HDMI


Pro AV Chain (HDMI/DisplayPort)

Production Switcher

Microphones

Cameras Video/Audio Server

BluRay

PC

Interactive Display

LED Wall

Digital Signage

HDMI/DP Switch

Retail

Education

Live Events

Conferencing

Corporate

Control Room

KVM

Video Wall Controller

Projectors

BluRay

SDI

DisplayPort or HDMI

Ethernet


Key Pro AV & Broadcast Applications

•Audio Processor

•Dolby Encoder

•Intercom & Talkback

•Audio over IP

Audio Systems

•Distribution Encoder

•Contribution Encoder

•Integrated Receiver Decoder (IRD)

Encoders & Decoders

•LED Video Wall

•Projector

•Digital Signage

•Multiviewer

Monitors & Projection

•Non Linear Editing (NLE)

•Digital Video Effects (DVE)

•Up/Down/Cross Conversion

Video Processing Cards

•Studio Camera

•ENG Camcorder

•Cinema Camera

•Camera Control Unit (CCU)

Pro Cameras & Camcorders

•Router, Switch Matrix

•Production Switcher

•Master Control

•Vision Mixer

Routers & Switchers

•Video Server

•Channel-in-a-box

•Cloud Accelerator

•Media Players/Storage

Servers & Storage

•Video over IP Gateway

•Mini Converters

•Network Monitoring

•KVM

Networking & Video Converters

•Telepresence

•MCU

•Endpoint

Video Conferencing

•DVB Transmitter

•ATSC Modulator

•ISDB Headend

•DMB Tester

Terrestrial & Satellite Modulators


Industry EvolutionsDriving insatiable bandwidth and new connectivity standards

Faster PixelsBetter PixelsMore Pixels

4K2K in main stream

8K4K at 2020 Tokyo Olympics

– will need 100+ Gbps

bandwidth Higher Dynamic Range (HDR)

Wider color space i.e. Rec 2020

Higher Frame Rate (HFR)

THE HOBBIT shot at 48 fps

instead of legacy 24 fps

Avatar-II to be shot at 60 fps

BBC research indicates 120 fps

required for UHD experience


Video System Design Challenge

Memory

Video Processing• Scaling • warping• stitching• resampling• Encoding• ……

Video Processing• blending• color correction• Decoding• …….

Video memory controller4K: 28+ Gbps8K: 100+ Gbps

4K: nx28+ Gbps8K: nx100+ Gbps

HDMI

DP

HDCP

UHD-SDI

MIPI

HDMI

DP

UHD-SDI

MIPI

Vx1

4K HP = 1.188 Giga Pixels/second8K HP = 4.752 Giga Pixels/second

4K: 28+ Gbps8K: 100+ Gbps


Great SDI Jitter Performance

Real Time Video Processing

Bandwidth for Video Buffering

Highest Video Quality

Support New Standards

Reconfigurable & Scalable

Manage Power & Cost

Bridge Comms & Broadcast

The Value of Xilinx Devices in Broadcast & Pro A/V


Video Interfaces


Device 7-Series UltraScale UltraScale+

ProtocolArtix

GTP

Zynq

GTP

Kintex

GTX

Zynq

GTX

Virtex

GTX

Virtex

GTH

Virtex

GTZ

Kintex

GTH

Kintex GTY Virtex

GTH

Virtex

GTY

KU+ GTH ZU+ GTH

HDMI 2.0a/2.0/1.4

DisplayPort 1.2a

UHD-SDI (SD/HD/3G/ 6G/12G)

MIPI DPHY Through external components Through Native HPIO

Video Interface Support Matrix

12G-SDI will require -3 speed grade Kintex 7

6G/3G-SDI can be implemented in Kintex7 -1 speed grade

Artix-7 HDMI/DP/SDI support available through 3rd party

DP and HDMI designs for ZU+ coming in 2017.3


Video Interfaces - MIPI


Xilinx MIPI Solutions Overview

Xilinx offers MIPI based solutions for camera

sensor capture and driving displays

– Relevant standards are MIPI-CSI2 and DSI

CSI2 and DSI require DPHY IO standard to

implement the physical layer

– DPHY IO switch between low-power mode ( 0- 1.2V single

ended) and High-speed mode ( 0.1 - 0.3V differential) on

the same pin

Implementing DPHY with Xilinx FPGAs

– 20-nm and earlier devices do not support DPHY, Use

• Low cost solution using external resistors (XAPP894)

• “OR” High performance using Meticom FPGA bridge chips

– 16-nm devices have built-in support for DPHY physical layer

• No need of external bridges or resistor networks

Implementing CSI2 and DSI controllers on Xilinx

FPGA

– Users have two options

• Xilinx’s in-house IP ( Except DSI RX)

• Northwest Logic IP

MIPI-CSI2

MIPI-DSI


Spartan-6 Artix-7 Kintex-7 Virtex-7 Zynq-7000

Kintex /

Virtex

UltraScale

UltraScale+

(Zynq

MPSoC)

MIPI D-PHY External

D-PHY chipor XAPP894

External D-PHY chipor XAPP894





Native IO support

MIPI DSI / Host (TX) IP Provider

NWL XilinxNWL

XilinxNWL

XilinxNWL

XilinxNWL

NWLXilinxNWL

(Roadmap)MIPI DSI / Peripherals (RX)IP Provider

NWL NWL NWLNWL Xilinx

NWL NWLNWL

(Roadmap)

MIPI CSI2 / RXIP provider

NWL XilinxNWL

XilinxNWL

NWLXilinxNWL

NWLXilinxNWL

(Roadmap)

MIPI CSI2 / TXIP provider

NWL XilinxNWL

XilinxNWL

NWL Xilinx

XilinxNWL

NWLXilinxNWL

(Roadmap)

MIPI Solution Selection Matrix

Recommended


DPHY IO Support Mode

Compatible DPHY IO

• External passive network required for the level shit, detailed description referred to XAPP894.

• Limited performance up to 800 Mbps @ < 300mm trace length, and additional board level SI simulation is highly recommended.

• Applicable to all Xilinx 28nm and 20nm FPGAs and SoCs

Compliant DPHY IO

• External chip(MC20001/MC20002) required to support full performance up to 1.5Gbps.

• Applicable to all Xilinx 28nm and 20nm FPGAs and SoCs.

Native DPHY IO

• No external component required on board design, line rate range is 80M~1500M bps.

• Dedicate DPHY I/O BUF and DCI instantiated on HP I/O Bank.

• Applicable to Xilinx 16nm FPGAs and SoCs only

FPGA

MIPITransmitter

MIPIReceiver

HS

LP

HSTL

LVCMOS

150 Ohm

150 Ohm

60 Ohm

60 Ohm

120 Ohm

120 Ohm

1.8V I/O Bank

MIPITransmitter

MIPIReceiver

LP

LVDS

HSUL

100 Ohm

100 Ohm

150 Ohm

FPGA

HS

1.8V I/O Bank

MIPI Transmitter MIPI Receiver

MIPI Transmitter MIPI Receiver


Designed and offered by Fidus/inrevium

– Ordering code TB-FMCL-MIPI for 7-series devices

– Ordering code TB-FMCL-MIPI-BYPASS for US+ Device

• External D-PHY bridge chips are bypassed

– Contact Inrevium for boards

2x MIPI Ports configurable as

– 2x 4 lane CSI2

– 2x 4 lane DSI

– 1x CSI2 and 1x DSI

Reference design and demo available

– Show cases MIPI CSI RX and DSI TX

Additional components required for demo

– OV13850 Camera module and adapter from Inrevium

– AUO display module and adapter from Inrevium

– Contact PM for loaner Demo HW

Reference design ported to IMX274 platform

in Vivado 2017.3

– Platform shared with Embedded Vision Reference designs

– Lower cost FMC ($299)

MIPI FMC Hardware MIPI FMC Block Diagram

MIPI FMC and Camera kit

mailto:[email protected]?subject=MIPI BYPASS FMC boards

http://shop.leopardimaging.com/product.sc?productId=267


OV13850

MIPI end to end system with 4K support

– ZCU102 (Vivado 2016.4) + TB-FMCH-MIPI-ByPASS

• OV13850 4K Camera Module

• Multiple data format (RAW8, RAW10, RAW12)

support

• Multiple Pixel mode support, multiple resolution

(720P, 1080P, 4K)

– HDMI, DSI as display options

DSI Display Panel

MIPI

CSI2

Rx SS

Color

Filter

ArrayVDMA1 VDMA2

DDR

Single pixel mode

Test

Pattern

Generat

or

AXIS Broadca

ster

Dual/Quad pixels/clock

Pass Through/

Generate mode

HDMI

Tx SS

VPHY Control

ler

DSI Tx

SS

HDMI Monitor

Video

Scaler

AUOS

DSI Display

Xilinx MIPI Reference Design (XAPP1302)

Vivado 2017.3 Public Access

– Design will be ported Lower cost EV FMCs from

Leopard Imaging

– Sony IMX274 Platforms

– Will include Xilinx in-house ISP supporting 4K60

– Will also be ported to Linux


Video Interfaces - SDI


7-series UltraScale UltraScale+

GTP GTX GTH GTH GTY GTH GTY

3G-SDI XAPP1097 XAPP592 XAPP1187 XAPP 1290 No Support* 2017.1

(subsystem)

No Support*

UHD-SDI

(12G-SDI)

No Support XAPP 1249 No Support XAPP 1248 No Support* 2017.1

(subsystem)

No Support*

UHD-SDI Subsystem

2017.1 Introduction of SDI subsystems

– Subsystem will include GT wrapper, AXI bridges(optional) and AXI lite interface(optional)

• Initial rollout will only support AXI bridges interface

– Not in 2017.1 IP catalog; Offered as Local repo

– In the form of Pass Through Example design on ZCU102 + TB-12G-SDI FMC

– Will support US+ GTH Only

– For 7-series and US, continue to use current XAPP reference designs

Recommended SDI support by GT


Video Interfaces - HDMI


Xilinx HDMI Solution

HDMI TX and RX Subsystems

– Designed to support HDMI1.4 and 2.0 specifications

– RGB, YUV

– 8,10,12,16 bpc

– Audio up to 8 channels

– Side band communication thru DDC or SCDC

– Device and GT Support

• US+ GTH ( pre-production)

• US GTH

• 7-Series GTX

• Artix 7 GTP ( Limited to HDMI 1.4)

– HDCP 1.4 and 2.2

– Design Example in Vivado supporting

• KC705

• KCU105

• ZC706

HDMI Block Diagram

Function LUT FF RAM GT

HDMI 2.0/1.4 TX 7500 9650 2

HDMI 2.0/1.4 RX 13800 11300 2Video PHY + NI-DRU

( common for RX and TX) 8000 8000 3

HDCP 2.2 TX 11674 6397 0

HDCP 2.2 RX 13140 7676 0

HDCP 1.4 TX 3724 3346 2

HDCP 1.4 RX 3322 3282 2

Resource Estimate for use case : HDMI2.0, 16bpc, 4 pixels/clock

Video

PHY


Feature Time Line

Video resolutions up to 4K @ 60 fps, 444 ( UHD and DC4K) Now

Video format RGB444, YUV 444, 422, 420 Now

Deep color support ( TX) Now

Audio 8 channel (IEC 60958) Now

Data Display Channel (DDC) and Status and control data channel (SCDC for HDMI2.0) Now

Info frames Now

Hot plug / EDID Now

GT Support: US+ GTH, UltraScale GTH, 7-series GTX and GTP Now

HDCP 1.4/2.2 Now

3D video timing Now

HDMI2.0a ( HDR) Now

HDCP 1.4/2.2 Repeater/Converter Now

V4L2 and DRM drivers Now

HDMI Subsystem Feature Summary


Board Guidelines

RX

– GTs don’t support TMDS level signaling

– TMDS levels emulated using external

resistors of 50 ohms• Located close to FPGA GTs

– Use of external TMDS EQ/repeater chip

recommended for PHY compliance and

HDMI 2.0 data rates • TI’ TMDS181

• Parade Technology’s PS8409

– Refer to ZCU106 HDMI schematics

TX

– GTs don’t support TMDS level

signaling

– TMDS level Shifting done using

external level shifter ASSPs

• TI’ SN65DP159

• Parade Technology’s PS8409

– Refer to ZCU106 HDMI Schematics

Intel PSG (Altera) also requires EQ and Redriver on RX and TX respectively


HDMI & HDCP Licensing


HDMI Collateral

HDMI Documentation

– HDMI TX Subsystem and HDMI RX

Subsystem Product guides

– HDCP user guide

– Video Phy user guide

HDMI Design Example (XAPP1287)

– Targets KC705 with HDMI FMC

– Pass Thru design from RX -> TX

– TX only option with Test Pattern Gen

– Optional HDCP

– Up to 4K60

– UART based debug terminal

– Support for KCU105 and ZC706

examples design in Vivado IP flow

TB-FMCH-HDMI4K

Supplied from

Inrevium/Fidus

Example Designs in Vivado

supporting:

KC705, ZC706, KCU105

https://www.xilinx.com/support/documentation/ip_documentation/v_hdmi_tx_ss/v2_0/pg235-v-hdmi-tx-ss.pdf

https://www.xilinx.com/support/documentation/ip_documentation/v_hdmi_rx_ss/v2_0/pg236-v-hdmi-rx-ss.pdf

http://www.xilinx.com/support/documentation/ip_documentation/hdcp/v1_0/pg224-hdcp.pdf

http://www.xilinx.com/support/documentation/ip_documentation/vid_phy_controller/v2_0/pg230-vid-phy-controller.pdf

https://www.xilinx.com/support/documentation/application_notes/xapp1287-hdmi-on-fpga-gtx-transceivers.pdf


Magewell HDMI 2.0 IP Solution- Company Overview

About Magewell

– Founded in 2011, headquarter in Nanjing, China

– Focused on the core technology of audio and high-quality video processing.

– Main product includes PCIe video capture card and USB 3.0 capture boxes,

support multiple video interface like SDI, DVI, HDMI, and up to 4K support.


Magewell HDMI 2.0 IP Solution- IP Feature set

HDMI 2.0 IP– DVI, HDMI 1.4 and HDMI 2.0 Compatible

– Support RGB, YCbCr

– 4:2:0, 4:2:2, 4:4:4 resampler

– 1, 2, or 4 symbol/pixel per clock input

– 8, 10, 12, and 16-bit Deep-color support

– 2 to 8 audio channels

– Support customized resolution, eg. 8K x 1K

– Support InfoFrame

– HDMI RX support EDID, SCDC

– Integrated Video Timing Counter and Video Timing Generator

– Flexible clocking structure, support source clock or local reference clock

– MCU register interface for configuration

– Small footprint

• 18K LUT only (for 4 pixel, 4Kx4K, RX+TX+PHY, full feature) to fit into 7A35T

– Target device and GT

• Artix-7 GTP (including related Zynq-7000)

• Kintex-7 GTX (including related Zynq-7000)


Magewell HDMI 2.0 IP Solution- Evaluation and Architecture

Hardware evaluation platform

Software GUI for evaluation

Control stream

Data stream


Video Interfaces - Future


DisplayPort 1.4 HDMI2.1( Not final)

Max BW(raw) 32.4 Gb/s (8.1 x4) 48 Gb/s over 4 lanes

Major Features • 8K30,5K60,4K120

• Dynamic HDR, deep

colorimetry

• DSC, connector compression

for higher resolutions and

frame rates

• Same cable expected

• 8K60,4K120,

• Dynamic HDR, deep

colorimetry

• DSC, connector compression

for higher resolutions and

frame rates – 10K at 120Hz

• Newer cable is required

Products in Market GPUs: Nvidia 1080, 1050;

AMD Radeon

None known

Next Gen Video Interfaces with 8K support

On Xilinx Roadmap


Video Processing


Xilinx Video Processing

Subsystem

Xilinx Video Mixer

LogiCore

Omnitek Scalable Video

Processor (OSVP)

Omnitek Video Warp

Processor

Features• Up to 4K60 format conversion• YUV and RGB in 4:2:0, 4:2:2 or 4:4:4

format; 8-, 10, 12 and 16-bit• Configurable as Deinterlacer or Scaler

or Color Space Converter or Chroma Resampler

• Includes VDMA for frame buffer and frame rate conversion

• Option for Memory less configuration • AXI Interfaces allow easy integration

with HDMI and DP subsystems

*Replaces the legacy VIPP cores

• Allows mixing of up to 8 layers• Up to 4K60 on each layer• Support for memory based on

streaming layers• Additional Logo insertion layer• Optional scaling on each layer• Per Pixel/Layer alpha blending• RGB and YCbCr color spaces

*Replaces the legacy VIPP cores

• Up/Down/Cross Conversion• Input up to 4K60; output up to 4K120

with frame synchronization• YUV and RGB in 4:2:0, 4:2:2 or 4:4:4

format; 8-, 10- or 12-bit• Interlaced, progressive or segmented

frame (PsF)• Crop and resize with Super-

Resolution image enhancement• Motion/Edge-adaptive deinterlacer• 6-axis YUV/RGB colour correction• 3:2 & 2:2 cadence processing• Alpha blending of multiple sources• Up to 8 video processing paths

• Fisheye, Keystone, Rotation, Perspective mapping

• Arbitrary Warps within a 0.5x to 2x scaling limit

• Provides support for up to 3840*2160@60Hz or 2712*1528@120Hz

• Option to support multiple lower resolution inputs

• Low latency (1/6th to 1 frame)• Control software for OSD

wireframe and mesh calculation• 10-bit 4:4:4 processing

Device

support

7-Series• All 7-series devicesUltraScale• All UltraScale devicesUltraScale+ • Zynq

7-Series• All 7-series devicesUltraScale• All UltraScale devicesUltraScale+ • Zynq

7-Series• All 7-series devicesUltraScale• All UltraScale devices

7-Series• All 7-series devicesUltraScale• All UltraScale devices

Related Doc

/Example

designs

PG231XAPP1291XAPP1285

PG243XAPP1291

XAPP1264 Contact Omnitek(works with RTVE)

Licensing

information

EF-DI-VPSS EF-DI-VID-MIX Contact Omnitek Contact Omnitek

Video Processing Solutions Selection Matrix

Xilinx in-house solution Xilinx partner IP with advanced feature set

http://www.xilinx.com/support/documentation/ip_documentation/v_proc_ss/v2_0/pg231-v-proc-ss.pdf

http://www.xilinx.com/support/documentation/application_notes/xapp1285-scaling-video-processing-with-vpss.pdf

http://www.xilinx.com/support/documentation/ip_documentation/v_mix/v1_0/pg243-v-mix.pdf

http://www.xilinx.com/support/documentation/application_notes/xapp1264-rtve3-1.pdf


Video Processing – Xilinx VPSS


Video Processing Subsystem

Modes of operation

– Full-format conversion mode

– Scaler only mode

– Memory less configuration

– Can be configured to perform function of any

one sub cores

Offers out of the box 4K format

conversion

– Up/down/cross conversion from 480i/p to 2160P

– All common video formats supported

– High-level subsystem API abstracts from details

under the hood

300MHz Fmax on Kintex -2 and

equivalent devices

– 1/2/4 Pixels per clock

– Can support 4K60 on lower speed grade devices

using 4 pixels per clock

–

Motion Adaptive/

Spatial

Deinterlacer

Scaler

( 4k, bi-linear, bi-

cubic or

polyphase)

Chroma

resampler

(420 <-> 444)

Color matrix

( RGB <-> YUV)

Video Processing Subsystem

Up to 4K60

RGB/YUV

444, 422,420

8,10,12,16bpc

Up to 4K60

RGB/YUV

444, 422,420

8,10,12,16bpc

External Mem interface thru VDMA

subcore subcore subcoresubcore

Letter Box

subcore


Video Mixer Subsystem

Supports up to 4k60, upgradeable

to 8K

Up to 8 layers, with additional logo

layer (in BRAM)

Layers can be either streaming or

memory (built-in DMA)

Per layer global alpha blending

Per Pixel Alpha on memory layers

Scaling per layer

– pixel/line repeat

Supports RGB, YUV 4:4:4 and 4:2:2

color spaces up to 16 bits per

component

– Internal color space conversion and

chroma resampling done when needed

Optional Chroma

Resampling

Optional Color Space Conversion

Optional Chroma

Resampling


Mix

Optional Chroma

Resampling


Mix

Optional Chroma

Resampling


MixOptional

Color Space Conversion

Optional Chroma

Resampling

Four Layer Mixer block diagram


Video Processing Sub-System Ref Design (XAPP1291)

Showcases

– VPSS, Video mixer

– Integrates HDMI subsystem for

capture and display

On Xilinx.com in July 2016

Features

– UP/Down/Cross conversion up to

4K using VPSS

– Video Mixer with 4 layer mixing

and logo insertion

– Run-time configuration through SW

application

4K Monitor

KC705 Board

HDMI RxVideo Processing

SubsystemHDMI TxVideo Mixer

HDMI Rx Stream

MixerLayer 1

MixerLogo Layer

MixerLayer 2

MixerLayer 3


Video Processing – Omnitek RTVE


4K/8K Real Time Video Engine (RTVE)

OmnitekXilinx Inrevium

Omnitek RTVEDeinterlacing

Scaling/OverlayingISP/Warp/Rotate

Stitching

MIPICSI2

Cro

ssp

oin

t

HDMI2.0

DP1.2

UHD-SDI6G & 12G

Cro

ssp

oin

tMIPIDSI

HDMI2.0

DP1.2

UHD-SDI6G & 12G

Linux O/SWeb Server

RTVE GUIGraphics

DDR3 Memory

HDCP V-by-One HS

Reference Design Software


Features Device / Demo Board

Availability

RTVE 2.1 Six-channel HD video processing:• format cross conversion• 2D Graphics• 3GSDI & HDMI 1.4b

XC7Z045 / OZ745 CY2013 - Now

RTVE 3.1 Dual-channel 4k video processing:• SD-4K format cross conversion• 3G/6G/12G SDI • DP 1.2a & HDMI 1.4b

XC7Z045 / OZ745 CY2014 - Now

RTVE 3.5 Single Channel 8K video processing:• SD-8K format cross conversion• 1x 12G SDI input• 4x DP 1.2a output

XCKU060 / ACDC-8K

CY2015 - Now

RTVE 4.0 Single-channel 4K arbitrary warp:• Integrated format cross conversion• warping/dewarping• 3G / 6G / 12G SDI / HDMI 1.4b / DP 1.2a

XC7Z045 / OZ745 CY2016 - Now

RTVE 4.1 4x 1080p60 to 1x 4K60p Live Video Stitching:• 4x 3G SDI Input / 1x DP 1.2 Output• Real time warping & dewarping• Real time stitching and auto calibration

XC7045 / OZ745 CY2016 Now

RTVE 4.2 4x 1080p60 to 1x 4K60p Live Image Stitching:• 4x FHD MIPI Inputs / 1x DP 1.2 Output• Real time ISP, warping & stitching• Real video compression & streaming

XCZU7EV / ZCU106

2017 Q2

RTVE Revisions & Roadmap

Focus on video format cross conversion & multi-viewing

Focus on image manipulation &

panoramic stitching


RTVE 3.5: SD↔8K Format Cross Conversion in KU060

Full Scale Video Processing Coverage from SD to 8K – Available Now


Omnitek Video Warp Processor

www.omnitek.tv


RTVE 4.1: 4x 1080p60 1x 4K60p Panoramic View in 7Z045

Hardware demonstration in ISE 2017 – Available Now


RTVE 4.2: ISP & VCU Integration in ZYNQ UltraScale+ MPSOC

Hardware demonstration target: 2017 Q2


Video Codec


Zynq UltraScale+ MPSoC Video Codec Unit


Zynq UltraScale+ MPSoC VCU

Video

Encoder

Video

Decoder

VCU

Power &

Performance

Data

Control

Control

Data

Control


MPSoC VCU: Encoder Features

Feature H.265 (HEVC) H.264 (AVC)

General Offline Processor

Separate Encoder/Decoder Engines -> Simultaneous Encode and Decode

Power Management and Performance Monitoring

Profiles Main,

Main Intra,

Main10,

Main10 Intra,

Main 4:2:2 10, Main 4:2:2 10 Intra

Baseline,

Main,

High,

High10,

High 4:2:2

Level up to 5.1 High Tier up to 5.2

Sample bit Depth 8 to 10 bpc

Chroma format YCbCr 4:2:0, 4:2:2, 4:0:0

Slice Type I, P, B

Video Format Progressive only

Pixel Format Semi Planar only

Entropy encoding CABAC CABAC, CAVLC

Slice Support Supported Supported

Tile Support Supported Not Applicable

In-loop deblocking filter Supported Supported

Configurable bit rate up to 533 Mbit/s up to 960 Mbit/s

Rate control CBR, VBR, Constant QP


MPSoC VCU: Decoder Features

Feature H.265 (HEVC) H.264 (AVC)

General Offline Processor

Separate Encoder/Decoder Engines -> Simultaneous Encode and Decode

Power Management and Performance Monitoring

Profiles Main,

Main Intra,

Main10,

Main10 Intra,

Main 4:2:2 10,

Main 4:2:2 10 Intra

Baseline,

Main,

High,

High10,

High 4:2:2

Level up to 5.1 High Tier up to 5.2

Sample bit Depth 8 to 10 bpc

Chroma format YCbCr 4:2:0, 4:2:2, 4:0:0

Output Pixel format Semi-Planar only

Slice Type I, P, B

Format Progressive only

Configurable bit rate up to 533 Mbit/s up to 960 Mbit/s

Rate control CBR, VBR, Constant QP

Error resilience/concealment Supported

Trick Mode Supported


Video Codec - Hardware


Video Encoder Architecture


Video Encoder Use Case

Encoder

VideoOptional

Encoder

Buffer


Video Decoder Architecture


Video Decoder Use Case

Decoder

Video


Video Codec - Software


VCU Linux Software Stack

HardwareVCU Hardware

Kernel Driver

(ioctl)

Encoder IP

MCU Encoder core +

Decoder IP

MCU Decoder core +

Control Software APIs

Open Max Implementation Layer

(OMX IL)

gstreamer

gstreamer OMX IL plugin

User space

Application

Kernel space


Zynq UltraScale+ MPSoC: Gstreamer Framework


Video Pipeline Buffer Management: Zero Copy


Video Codec – Reference Design


Starting platform upon which users may implement their own VCU based

design

Enables evaluation of the VCU performance and its capabilities

– User selectable compression, stream count, video source, etc.

– Both visual & analytical analysis options

PL based video pipeline supporting:

– 4K or 2K video input sources

– Multi-stream encoding and decoding supported

– VCU Compression

– VCU Decompression

– 4K output to either DP and HDMI

Video pipeline controlled via Linux

Delivered as a Reference Design with associated documentation

– Same format as the ZU+ Base TRD

VCU Targeted Reference Design Overview


Based 2017.1 – May 31st

– HDMI In

– PS Display Port out

– 4KP30 single stream, 8 bit, 4:2:0

– USB/SATA/GE/SD file input only

Based on 2017.3 – Release plus 6 weeks

– HDMI In/Out

– SDI In/Out

– MIPI In/Out – TBD

– PS Display Port out

– 4KP60 single stream, 10 bit, 4:2:2

– Two 4KP30 streams in, one HDMI and one Test Pattern Generator

– Multiple streams

TRD Modular Approach


Encoder

HDMI-RXVCU modules

ZCU 106 Board

DP

Subsystem

HP2

Frame

Buffer

Write

GPU

USB

SATA

DDR

C

SD

APU

AXI Interconnect

Processing System

Video In

HPC0HP0HPM0 HP1

AXI Interconnect

Frame

Buffer

Write

VPSS TPG

Capture

MCU

Decoder

MCU

Register Space

AXI

Interconnect

AXI

Interconnect

MIG

AXI Interconnect

FB

Read

VPSS

Video Mixer

HDMII-TX

FB

Read

VPSS

FB

Read

VPSS

4K 60P Display

DDR

DDR

Programmable Logic

Software Stack

• Linux

• Gstreamer

• OpenMax

• Control Software

• MCU Firmware

2017.1 TRD


Encoder

HDMII-RXVCU modules

ZCU 106 Board

DP

Subsystem

HP2

Frame

Buffer

Write

GPU

USB

SATA

DDR

C

SD

APU

AXI Interconnect

Processing System

Video In

HPC0HP0HPM0 HP1

AXI Interconnect

Frame

Buffer

Write

VPSS TPG

Capture

MCU

Decoder

MCU

Register Space

AXI

Interconnect

AXI

Interconnect

MIG

AXI Interconnect

FB

Read

VPSS

Video Mixer

HDMII-TX

FB

Read

VPSS

FB

Read

VPSS

4K 60P Display

DDR

DDR

Programmable Logic

Software Stack

• Linux

• Gstreamer

• OpenMax

• Control Software

• MCU Firmware

2017.3 TRD

xilinx standard templatesdi/aes router ird microphones camera intercomm sync pulse generator...

Documents