從音樂走向影音服務 - kkbox 的影音之路奮鬥史 - 序章
Post on 22-Jan-2017
7.994 Views
Preview:
TRANSCRIPT
���������KKBOX ������ - ��
Drake (���)Sr. Principal EngineerHead of Video Content PlatformKKBOX
by Shuen-Huei Guan, KKBOX. 2015 1
Who am I?
• administrator, Ptt BBS
• technical director / R&D manager, Digimax Inc.
• team player, KKBOX Technologies Ltd.
• contributor, PyCon Taiwan
• phd candidate, GINM, CISE, NTU
by Shuen-Huei Guan, KKBOX. 2015 2
• VideoPass / (ongoing) AnimePass video services in Japan.
• Service up to ~1M users.
• KKBOX Video team: ~40 in one year.
• Cooperation with Japanese organizations.
• To keep growing and heading for the next!
by Shuen-Huei Guan, KKBOX. 2015 3
by Shuen-Huei Guan, KKBOX. 2015 4
What's a Streaming Service?
by Shuen-Huei Guan, KKBOX. 2015 5
What's a Streaming Service?
by Shuen-Huei Guan, KKBOX. 2015 6
Much complicated than just a data flow
• CMS for content management.
• Flexible system (we thought) for evolving business logics.
• System optimication for operation expense.
• Scalability! (we're in cloud era)
by Shuen-Huei Guan, KKBOX. 2015 7
We have done that in KKBOX,and it works (really well)!
by Shuen-Huei Guan, KKBOX. 2015 8
by Shuen-Huei Guan, KKBOX. 2015 9
350 team players to serve10M users across 6 countries
by Shuen-Huei Guan, KKBOX. 2015 10
20M songs
by Shuen-Huei Guan, KKBOX. 2015 11
O2O Events
by Shuen-Huei Guan, KKBOX. 2015 12
If we can make music streaming work, how
about video streaming?— KKBOX CxO
by Shuen-Huei Guan, KKBOX. 2015 13
A turning point: KDDI
by Shuen-Huei Guan, KKBOX. 2015 14
Inside News
• VideoPass/AnimePass switched to KKBOX.
• KDDI is a big stockholder in KKBOX.
• Japan-Taiwan is better than Japan-???.
by Shuen-Huei Guan, KKBOX. 2015 15
We thought it's a peace of cake!
by Shuen-Huei Guan, KKBOX. 2015 16
Let's work on a video-on-demand service
• Adaptive streaming.
• DRM protection.
• Video encoding on cloud.
• Totally cloud-based system.
by Shuen-Huei Guan, KKBOX. 2015 17
Adaptive streaming
refer: https://www.wikiwand.com/en/Adaptivebitratestreaming
by Shuen-Huei Guan, KKBOX. 2015 18
Adaptive streaming (cont'd)
refer: https://www.wikiwand.com/en/Adaptivebitratestreaming
by Shuen-Huei Guan, KKBOX. 2015 19
Adaptive streaming benefits
• An ongoing trend for video streaming.
• YouTube, Netflix, Amazon, HbbTV, ...
• Multiple standards.
• HLS, HDS, MSS, MPEG-DASH, ...
• Matured technology.
• HTTP, HTML5, H.264, AAC, ...
by Shuen-Huei Guan, KKBOX. 2015 20
Adaptive streaming benefits (cont'd)
• Friendly UX with mobile in-mind.
• 3G/4G/Wifi, Android/iOS
• Copyright done easily (and right?!).
• MS PlayReady, Google Widevine, Apple FairPlay, ...
• Scalability with good performance.
• CDN, HTTP/2 or QUIC, ...
by Shuen-Huei Guan, KKBOX. 2015 21
by Shuen-Huei Guan, KKBOX. 2015 22
Adaptive streaming is seizable, but DRM would give you lots of
nightmares.
by Shuen-Huei Guan, KKBOX. 2015 23
ref: https://www.wikiwand.com/en/Digitalrightsmanagement
by Shuen-Huei Guan, KKBOX. 2015 24
(Fortunately,) we have Common Encryption (CENC).
by Shuen-Huei Guan, KKBOX. 2015 25
DRM
• Microsoft PlayReady nearly dominates DRM market right now.
• Google Widevive has huge potential because of Android/Chrome.
• Apple FairPlay stands for her own.
by Shuen-Huei Guan, KKBOX. 2015 26
by Shuen-Huei Guan, KKBOX. 2015 27
Video Encoding Flow
by Shuen-Huei Guan, KKBOX. 2015 28
by Shuen-Huei Guan, KKBOX. 2015 29
Video Encoding Flow
• Source video -> playable adaptive packages.
• Leverage all mighty open-source projects.
• All processed on cloud with capability of dynamic flow!
by Shuen-Huei Guan, KKBOX. 2015 30
by Shuen-Huei Guan, KKBOX. 2015 31
Job script for a workflow
Job {KKBOX Convert Video} -subtasks { Task {Source Inspection} -cmds { Cmd { emilia verify -i s3://bucket/source.mp4 } }
Task {Transcode} --parallel -subtasks { Iterate i -from 0 -to 4 -by 1 -template { Task {Transcode Audio} -cmds { Cmd { ffmpeg -i s3://bucket/source.mp4 -o /tmp/converted_$i.mp4 } } } Iterate i -from 0 -to 8 -by 1 -template { Task {Transcode Video} -cmds { Cmd { ffmpeg -i s3://bucket/source.mp4 -o /tmp/converted_$i.mp4 } } } }
Task {Adaptive} -subtasks { Task {DASH} -subtasks { } Task {HLS} -subtasks { } Task {MSS} -subtasks { } }} -sequential 1
by Shuen-Huei Guan, KKBOX. 2015 32
Orchestrating the execution of workflows for media streaming
service and even morePyCon APAC 2015
by Shuen-Huei Guan, KKBOX. 2015 33
System Architecture
by Shuen-Huei Guan, KKBOX. 2015 34
by Shuen-Huei Guan, KKBOX. 2015 35
AWS services
• EC2 / ELB / S3
• RDS / ElastiCache
• SWF (Simple Workflow)
• SQS / SNS / SES
• Lambda
• CloudWatch
by Shuen-Huei Guan, KKBOX. 2015 36
System architecture
• Backend / CMS
• Laravel (PHP)
• MySQL / Elasticsearch
• Video Encoding System
• Python
by Shuen-Huei Guan, KKBOX. 2015 37
AWS expense and status:
Confidential but available on stage.
by Shuen-Huei Guan, KKBOX. 2015 38
Third-party services
• Pingdom
• Leanplum
• Slack
• JIRA
by Shuen-Huei Guan, KKBOX. 2015 39
by Shuen-Huei Guan, KKBOX. 2015 40
We love Slack.
• Server status / alarm.
• Video encoding status.
• JIRA ticket updates.
• Video quality checking status.
• ...
by Shuen-Huei Guan, KKBOX. 2015 41
But we were wrong. Way more challenges
are waiting for us.
by Shuen-Huei Guan, KKBOX. 2015 42
Technical challenges
• Video file size is much larger than music files.
• 2hr HD movie = 150 x “5min 192kbps music”
• Video file encoding takes much longer.
• 100x time consumption than music.
• Lots of files.
• 2hr HD movie ~= 20,000~30,000 files.
by Shuen-Huei Guan, KKBOX. 2015 43
Technical challenges (cont'd)
• H.264 is not that easy.
• Each H.264-claimed devices support differently.
• Adaptive players needs effrots.
• Smart adaptation, initial lag vs smooth playback, A/V sync, ...
• Video streaming protocols are much complicated.
• HLS, MSS, DASH with DRM(s)!by Shuen-Huei Guan, KKBOX. 2015 44
Business challenges
• Asian services tend to be complex than elegant, especially Japanese ones.
• Business logics changed weekly, but it's an OEM, right?
• OEM is not what KKBOX's good at.
• Programming is an art, so is communication!
by Shuen-Huei Guan, KKBOX. 2015 45
Integration challenges
• Communication in English is ok, but Japanes is better.
• Hard problems doesn't kill developers, but communication does.
• More than 5 vendors to cooperate with.
• Docs of vendors' API? Nearly none.
by Shuen-Huei Guan, KKBOX. 2015 46
top related