李仁杰/ riot games head of data science
Post on 12-Jan-2017
12.648 views
TRANSCRIPT
DATA SCIENCE
RENJIE LI
AT RIOT GAMES
07.16.2016
2009 LAUNCH
TEAM ORIENTED
ONLINE GAME
120+ CHAMPS
LIVE PLAYERS VS. LIVE PLAYERS
BASKETBALL LEAGUE OF LEGENDS
Players & Data W O R L D W I D E
Statistics released Jan 2014
67+ million monthly active players
500+ billion data points per day
26 petabytes data collected since beta
DATA SCIENCE AT RIOT GAMES
EMPOWER RIOTERS TO MAKE BETTER DATA POWERED PRODUCTS
TEAM MISSION
DATA SCIENCE AT RIOT GAMES
PLAYER FOCUSED
DATA INFORMED, NOT DATA DRIVEN
TEAM PHILOSOPHIES
DATA DRIVEN INFORMED DECISION MAKING
DATA DRIVEN
DATA INFORMED
DATA SCIENCE AT RIOT GAMES
Data Science
DATA SCIENCE AT RIOT GAMES
Data Science
Risk Marketing
Ecommerce
DATA SCIENCE AT RIOT GAMES
Data Science
Social Play
Risk Marketing
Ecommerce
DATA SCIENCE AT RIOT GAMES
Data Science
Social Play
Risk
Match Making
Marketing
Ecommerce
DATA SCIENCE AT RIOT GAMES
Data Science
Social Play
Risk
Match Making
Player Onboarding
Marketing
Ecommerce
DATA SCIENCE AT RIOT GAMES
Data Science
Social Play
AI
Risk
R & D
Match Making
Eco System Player
Support
Player Onboarding
Game Balance
Design
Marketing
Ecommerce
CASE STUDY #1
UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR
UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR
CHAMPION PLAY PATTERNS
UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR
CHAMPION PLAY PATTERNS
UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR
CHAMPION PLAY PATTERNS
TYPE A CHAMPION
TYPE B CHAMPION
TYPE C CHAMPION
TYPE E CHAMPION
TYPE F CHAMPION
TYPE G CHAMPION
TYPE D CHAMPION
7 KEY TYPES OF CHAMPIONS
OUR DESIGN PHILOSOPHY ALIGNS WITH PLAYERS PLAY STYLES
UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR
0%
10%
20%
30%
40%
50%
60%
Type A Champion
Type B Champion
Type C Champion
Type D Champion
Type E Champion
Type F Champion
Type G Champion
Player 1
Player 2
PLAYER SEGMENTATION BASED ON HOW THEY
PLAY EACH TYPE OF CHAMPION
UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR
PLAYER SEGMENTATION BASED ON HOW THEY
PLAY EACH TYPE OF CHAMPION
9 DIFFERENT CHAMPION PLAY BEHAVIOR
UNDERSTANDING PLAYERS’ IN-GAME BEHAVIOR
Type A Type B Type C Type D Type E Type F Type G
IDEAL VS. CURRENT
DISTRIBUTION OF CHAMPION
CLUSTERS
CURR
ENT
IDEA
L
OVERSERVED UNDERREPRESENTED
CASE STUDY #1 TAKEAWAYS
CERTAIN CHAMPION ARCHETYPES MAY BE UNDERREPRESENTED OR OVERSERVED
HOW DIFFERENTLY PLAYERS PLAY OUR CHAMPIONS
WHETHER OUR DESIGN PHILOSOPHY ALIGNS WITH PLAYERS’ PLAY STYLES
CHAMPION PLAY PATTERN MODEL HELPS US BETTER UNDERSTAND:
CASE STUDY #2
UNDERSTANDING PLAYER ENGAGEMENT
UNDERSTANDING PLAYER ENGAGEMENT
week 1 week 2 week 3 week 4 week 5 week 6 week 7 week 8 week 9 week 10
week 11
week 12
week 13
week 14
week 15
week 16
week 17
week 18
week 19
week 20
week 21
week 22
week 23
week 24
Actual
TOTA
L HOU
RS P
LAYE
D
EVEN
T B
EVEN
T C
EVEN
T D
EVEN
T E
EVEN
T A
UNDERSTANDING PLAYER ENGAGEMENT
week 1 week 2 week 3 week 4 week 5 week 6 week 7 week 8 week 9 week 10
week 11
week 12
week 13
week 14
week 15
week 16
week 17
week 18
week 19
week 20
week 21
week 22
week 23
week 24
Predicted
TOTA
L HOU
RS P
LAYE
D
EVEN
T B
EVEN
T C
EVEN
T D
EVEN
T E
EVEN
T A
UNDERSTANDING PLAYER ENGAGEMENT
week 1 week 2 week 3 week 4 week 5 week 6 week 7 week 8 week 9 week 10
week 11
week 12
week 13
week 14
week 15
week 16
week 17
week 18
week 19
week 20
week 21
week 22
week 23
week 24
Predicted
Actual
TOTA
L HOU
RS P
LAYE
D
EVEN
T B
EVEN
T C
EVEN
T D
EVEN
T E
EVEN
T A
CASE STUDY #2 TAKEAWAYS
PROVIDE INSIGHTS FOR STRATEGIC DECISION MAKING AND PLANNING
QUANTIFY EFFECT OF EVENTS AND CHANGES IN THE GAME
FIND AND UNDERSTAND IMPORTANT FACTORS THAT PLAYERS ARE INTERESTED IN
WEEKLY ENGAGEMENT PREDICTION MODEL HELPS US :
Production
Predictive
Diagnostic
Descriptive
CASE STUDY #3
PERSONALIZED RECOMMENDATION
PERSONALIZED RECOMMENDATION
MANY ENTERTAINMENT SERVICES AND ECOMMERCE COMPANIES
HAVE MADE RECOMMENDER SYSTEM A PROMINENT PART OF THEIR
WEBSITES
GOOD RECOMMENDATIONS ADD ANOTHER DIMENSION TO THE USER
EXPERIENCE AND BOOST CONTENT ENGAGEMENT
PERSONALIZED RECOMMENDATION
GOOGLE NEWS: RECOMMENDATIONS GENERATE 38% MORE CLICKTHROUGH
NETFLIX: 66% OF THE MOVIES WATCHED ARE RECOMMENDED
AMAZON: 35% SALES ARE FROM RECOMMENDATIONS
PERSONALIZED RECOMMENDATION
SIMILAR
RECOMMENDATION
The recommender system is a predictive model that generates recommendations based on similarities between users and items as well as user-item interaction.
PERSONALIZED RECOMMENDATION
PERSONALIZED RECOMMENDATION
PERSONALIZED RECOMMENDATION
CASE STUDY #4
INSTANT FEEDBACK SYSTEM
TRIBUNAL
NATURAL LANGUAGE PROCESSING
NATURAL LANGUAGE PROCESSING
INSTANT FEEDBACK SYSTEM
TRIBUNAL NATURAL LANGUAGE
PROCESSING +
INSTANT FEEDBACK SYSTEM
SUPPORTS 14 DIFFERENT LANGUAGES
INCREASES CHATLOG COVERAGE FROM 10% TO 100%
DECREASES DETECTION TIME FROM WEEKS to 15 MINS
DATA SCIENCE AT RIOT GAMES
PLAYER FOCUSED
DATA INFORMED, NOT DATA DRIVEN
TEAM PHILOSOPHIES
EMPOWER RIOTERS TO MAKE BETTER DATA POWERED PRODUCTS
TEAM MISSION
QUESTIONS?