從音樂走向影音服務 - kkbox 的影音之路奮鬥史 - 序章

��KKBOX �� - ��

Drake (��)Sr. Principal EngineerHead of Video Content PlatformKKBOX

by Shuen-Huei Guan, KKBOX. 2015 1

Who am I?

• administrator, Ptt BBS

• technical director / R&D manager, Digimax Inc.

• team player, KKBOX Technologies Ltd.

• contributor, PyCon Taiwan

• phd candidate, GINM, CISE, NTU


https://www.ptt.cc

http://www.digimax.com.tw

http://www.kkbox.com

http://pycon.tw

http://www.inm.ntu.edu.tw

• VideoPass / (ongoing) AnimePass video services in Japan.

• Service up to ~1M users.

• KKBOX Video team: ~40 in one year.

• Cooperation with Japanese organizations.

• To keep growing and heading for the next!


What's a Streaming Service?


Much complicated than just a data flow

• CMS for content management.

• Flexible system (we thought) for evolving business logics.

• System optimication for operation expense.

• Scalability! (we're in cloud era)


We have done that in KKBOX,and it works (really well)!


350 team players to serve10M users across 6 countries


20M songs


O2O Events


If we can make music streaming work, how

about video streaming?— KKBOX CxO


A turning point: KDDI


Inside News

• VideoPass/AnimePass switched to KKBOX.

• KDDI is a big stockholder in KKBOX.

• Japan-Taiwan is better than Japan-???.


We thought it's a peace of cake!


Let's work on a video-on-demand service

• Adaptive streaming.

• DRM protection.

• Video encoding on cloud.

• Totally cloud-based system.


Adaptive streaming

refer: https://www.wikiwand.com/en/Adaptivebitratestreaming


Adaptive streaming (cont'd)

refer: https://www.wikiwand.com/en/Adaptivebitratestreaming


Adaptive streaming benefits

• An ongoing trend for video streaming.

• YouTube, Netflix, Amazon, HbbTV, ...

• Multiple standards.

• HLS, HDS, MSS, MPEG-DASH, ...

• Matured technology.

• HTTP, HTML5, H.264, AAC, ...


Adaptive streaming benefits (cont'd)

• Friendly UX with mobile in-mind.

• 3G/4G/Wifi, Android/iOS

• Copyright done easily (and right?!).

• MS PlayReady, Google Widevine, Apple FairPlay, ...

• Scalability with good performance.

• CDN, HTTP/2 or QUIC, ...


Adaptive streaming is seizable, but DRM would give you lots of

nightmares.


ref: https://www.wikiwand.com/en/Digitalrightsmanagement


(Fortunately,) we have Common Encryption (CENC).


DRM

• Microsoft PlayReady nearly dominates DRM market right now.

• Google Widevive has huge potential because of Android/Chrome.

• Apple FairPlay stands for her own.


Video Encoding Flow


Video Encoding Flow

• Source video -> playable adaptive packages.

• Leverage all mighty open-source projects.

• All processed on cloud with capability of dynamic flow!


Job script for a workflow

Job {KKBOX Convert Video} -subtasks { Task {Source Inspection} -cmds { Cmd { emilia verify -i s3://bucket/source.mp4 } }

Task {Transcode} --parallel -subtasks { Iterate i -from 0 -to 4 -by 1 -template { Task {Transcode Audio} -cmds { Cmd { ffmpeg -i s3://bucket/source.mp4 -o /tmp/converted_$i.mp4 } } } Iterate i -from 0 -to 8 -by 1 -template { Task {Transcode Video} -cmds { Cmd { ffmpeg -i s3://bucket/source.mp4 -o /tmp/converted_$i.mp4 } } } }

Task {Adaptive} -subtasks { Task {DASH} -subtasks { } Task {HLS} -subtasks { } Task {MSS} -subtasks { } }} -sequential 1


Orchestrating the execution of workflows for media streaming

service and even morePyCon APAC 2015


www.slideshare.net/drakeguan/orchestrating-the-execution-of-workflows-for-media-streaming-service-and-even-more



System Architecture


AWS services

• EC2 / ELB / S3

• RDS / ElastiCache

• SWF (Simple Workflow)

• SQS / SNS / SES

• Lambda

• CloudWatch


System architecture

• Backend / CMS

• Laravel (PHP)

• MySQL / Elasticsearch

• Video Encoding System

• Python


AWS expense and status:

Confidential but available on stage.


Third-party services

• Pingdom

• Leanplum

• Slack

• JIRA


We love Slack.

• Server status / alarm.

• Video encoding status.

• JIRA ticket updates.

• Video quality checking status.

• ...


But we were wrong. Way more challenges

are waiting for us.


Technical challenges

• Video file size is much larger than music files.

• 2hr HD movie = 150 x “5min 192kbps music”

• Video file encoding takes much longer.

• 100x time consumption than music.

• Lots of files.

• 2hr HD movie ~= 20,000~30,000 files.


Technical challenges (cont'd)

• H.264 is not that easy.

• Each H.264-claimed devices support differently.

• Adaptive players needs effrots.

• Smart adaptation, initial lag vs smooth playback, A/V sync, ...

• Video streaming protocols are much complicated.

• HLS, MSS, DASH with DRM(s)!by Shuen-Huei Guan, KKBOX. 2015 44

Business challenges

• Asian services tend to be complex than elegant, especially Japanese ones.

• Business logics changed weekly, but it's an OEM, right?

• OEM is not what KKBOX's good at.

• Programming is an art, so is communication!


Integration challenges

• Communication in English is ok, but Japanes is better.

• Hard problems doesn't kill developers, but communication does.

• More than 5 vendors to cooperate with.

• Docs of vendors' API? Nearly none.


Thank you.http://about.me/drakeguan


http://about.me/drakeguan

從音樂走向影音服務 - kkbox 的影音之路奮鬥史 - 序章

Technology