ios 视频 app 开发实践

iOS APP iOS APP

iOS APP

iOS app

Remix app AVFoundation GPUImage iOS app

iOS AVFoundation

OpenGL

GPUImage OpenGL

VideoToolbox

C ffmpeg

iOS

AVFoundation ffmpeg ffmpeg

OpenGL GPUImage OpenGL

GPUImage OpenGL iOS

Video Toolbox AV

AVFoundation AVCaptureDevice

AVPlayerAVAudioPlayer

AVComposation

AVAssetWriterAVAssetReader

AVFoundation

AVFoundation API

AVFoundation API AVAudioPlayer AVAudioRecorder

https://developer.apple.com/reference/avfoundation/avcapturedevice

AVFoundation -

iOS ImagePicker

AVCMoFOutStillOutPreviewLayer

Capture Session Input Input

Output Input

AVFoundation -

AVCaptureVideoDataOutput

AVFoundation -

AVAsset

AVFoundation Asset AVAsset

AVFoundation -

AVAsset

AVURLAsset

AVAsset AVURLAsset

AVFoundation -

AVAsset

AVAssertTracks Transform Metadata

AVAsset Tracks

AVFoundation -

AVPlayer

AVPlayerItem AVAsset

AVPlayerLayer

AVPlayer Asset AVPlayerLater

AVFoundation - Plus

AVPlayer

AVPlayerItem AVAsset

AVPlayerLayerAVPlayerLayerAVPlayerLayerAVPlayerLayerAVPlayerLayerAVPlayerLayer

AVPlayer PlayerLayer 200%

--

2015 Http

AVFoundation -

AVPlayer

AVPlayerItem AVURLAsset

AVPlayerLayer

Local Http Server Video URL

B ijkplayer

AVFoundation -

AVPlayer

AVPlayerItem AVURLAsset

AVPlayerLayer

Local Http Server Video URL

Local Proxy Server

AVURLAsset Http

http://www.codeproject.com/Articles/875105/Audio-streaming-and-caching-in-iOS-using

https://github.com/renjithn/AVAssetResourceLoader-Video-Example

iOS 6 AVAssetResourceLoader

http://www.codeproject.com/Articles/875105/Audio-streaming-and-caching-in-iOS-usinghttp://www.codeproject.com/Articles/875105/Audio-streaming-and-caching-in-iOS-using

AVFoundation -

AVURLAsset

Video URL

AVAssetResourceLoader

AVAssetResourceLoadingRequest

AVAssetResourceLoaderDelegate

iOS 6 ResourceLoader

AVURLAsset URL ResourceLoader delegate URL

delegate delegate Request

demo

AVFoundation -

AVURLAsset

Video URL

AVAssetResourceLoader

AVAssetResourceLoadingRequest

AVAssetResourceLoaderDelegate

https://github.com/renjithn/AVAssetResourceLoader-Video-Example

iOS 6 ResourceLoader

AVURLAsset URL ResourceLoader delegate URL

delegate delegate Request

demo

AVFoundation -

AVAsset

AssetAsset AV

AVFoundation -

AVAsset

AVComposition

AVMutableComposition

Asset AVComxxxxxxx

Asset Asset

AVFoundation -

AVAsset AVMutableComposition Asset

Asset Asset Track track VCD track

Asset

Composition Asset

MutableComposition Composition

AVFoundation -

CompositionTrack video CompositionLayerInstructioninstruction VideoComposition Core Animation

AVFoundation -

AudioMixInputParameters AudioComposition audio pitch

AVFoundation -

AssetExportSession

Composition app

AssetVideoComposition Asset ExportSession

demo

demo

Core Video

iOS

Core Video -

Core Video Core Video AVFoundation

Core Video iOS Objective-C API C API

Core Video -

CMSampleBuffer

CVPixelBuffer

CVImageBuffer

buffer

CMSB Buffer CVImageBuffer

CVPixelBuffer CVImageBuffer CVPB CVIB

Core Video - Pixel Format32 ARGB k32ARGBPixelFormat

32 BGRA k32BGRAPixelFormat

YUV YUV 411

Core Video -

CVImageBuffer/CVPixelBuffer

WidthHeight Data Size

Bytes Per Row Base Address

buffer Byte Base Address

Core Video - Bitmap BGRA ARGB

http://neareal.com/470/

RGB 32ARGB 32BGRA32 AlphaRedGreenBlue

BGRA BGRA RGB

4 byte BGRABGRABGRA

0 Bytes Per Row

Core Video - YUV Y

U

V

YUV YUVYUVYCbCrYPbPr YUV

YUV YUV

YCbCr

PbPr SDTV HDTV

Core Video - YUV

YUV YUV UV Y YUV

iPhone

Core Video - YUV

UV

Core Video - YUV 444Y:U:V=4:4:4YUV

422Y:U:V=4:2:2YUV2:1

411Y:U:V=4:1:1YUV4:1

420Y:UV=4:2Y:U:V=4:1:1YUV2:1

http://www.codernote.top/blog/info/119

RGBA 32 bitYUV UV

YUV RGB BGRA

http://www.codernote.top/blog/info/119

Core Video -

x.mp4

AVPlayerItem AVPlayer

AVPlayerItemVideoOutput

CVPixelBuffer

CADisplayLink

Player Item

PlayerOutput buffer

DisplayLink Output buffer

DisplayLink View

Core Video - Plus

demo

demo

GPUImage

OpenGL iOS GPUImage

GPUImage -

Objective-C OpenGL

showcase

GPUImageToneCurveFilterGPUImage

Photoshop ACV

ACV

GPUImage

GPUImageFilter -

texture GPUImageFilter

texture OpenGL texture

GPUImageFilter -

GPUImage shader

shadertoy texture

GPUImageFilter -

shader run

shader language c

highp vec4 4

GPUImageFilter -

GPUImageTwoInputFilterGPUImage

texture texture

texture

GPUImageTwoInputFilter

GPUImageTwoInputFilterGPUImage

texture texture

texture

texture1

texture2

GPUImageTwoInputFilter

demo

gpuimage demo

GPUImageFilter - Advanced

x.mp4

AVPlayerItem AVPlayer

AVPlayerItemVideoOutput

CVPixelBuffer

CADisplayLink

Filter

CVPixelBuffer

CVPixelBuffer AVPlayerItemVideoOutput

AVCaptureVideoDataOutput

AVAssetWriterInputPixelBufferAdaptor

buffer AVAssetWriterInputPixelBufferAdaptor pixel buffer SDAVAssetExportSession export session composition instructions pixel buffer

demo

demo demo

AVFoundation

Core Video

GPUImage OpenGL

SDAVAssetExportSession

SCRecorder

iOS

iOS

GPUImage CPU CPU OpenGL

OpenGL

Shader Language

Video Toolbox / ffmpeg

librtmp

VideoToolbox

ios 视频 app 开发实践

Mobile