predicting the “stars of tomorrow” on social media
TRANSCRIPT
-
Predicting the Stars of Tomorrow on Social Media
Wen-Huang Cheng ()
Multimedia Computing Lab (MCLab)Research Center for Information Technology Innovation (CITI),
Academia Sinica, Taipei, [email protected]
Presented at on 10 May 2017
-
2
-
Academia Sinica ()
The highest national research institute in Taiwan with about 1,000 professors (60 in EE/CS)
3
located in Nangang, Taipei
-
Multimedia Computing Lab (MCLab)
4
http://mclab.citi.sinica.edu.tw
-
We are social
5
Real World Digital World
-
Nanit Baby Monitor
-
9
Social Signals
-
Leading Social Networks
13[Ref] http://www.smartinsights.com/social-media-marketing/social-media-strategy/new-global-social-media-research/
-
14
-
Sociology and Human Interaction With the huge number of people who are involved nowadays with
social networks, it is very interesting to note how they are influenced by each other in many different ways. e.g., identity in the age of social media
15[Ref] http://edition.cnn.com/2015/10/05/health/being-13-teens-social-media-study/index.html
-
100 years after
100 years ago
-
Social Popularity Prediction General Popularity Prediction: Predicting the popularity
score of a new social media post by combining post content (photo, text or video) and user cues
17
Score: 4.9
?
Model
A new post
Predicted Popularity
Training Images
5.6 2.3
5.1 2.8
7.8 3.1
History data
-
Why is it important? wide applications and high business value
e.g., predicting the Stars of Tomorrow (top popular models) within the fashion Industry using social media
18[Ref] Style in the Age of Instagram: Predicting Success within the Fashion Industry using Social Media, CSCW 2016.
Fashion Model Directory (FMD) profile page
Can you tell who will be the top?
-
People are desired for knowing the future
19
[Ref] https://www.oreilly.com/ideas/inside-the-washington-posts-popularity-prediction-experiment
-
20
https://tianchi.shuju.aliyun.com/competition/index.htm
-
21
-
22
7
-
23
-
Our Related Publications Sequential Prediction of Social Media Popularity with
Deep Temporal Context Networks, IJCAI 2017. Time Matters: Multi-scale Temporalization of Social
Media Popularity, ACM Multimedia 2016 (full paper). Unfolding Temporal Dynamics: Predicting Social Media
Popularity Using Multi-scale Temporal Decomposition, AAAI 2016.
SocialCRC: Enabling Socially-Consensual Rendezvous Coordination by Mobile Phones, Pervasive and Mobile Computing, 2016.
24
-
What Makes A Post Popular?
25
[Ref]What Makes an Image Popular? WWW, 2014.
-
What Makes A Post Popular?
Features for prediction Post content
e.g., visual sentiment features (color and texture)
26
[Ref]Analyzing and predicting sentiment of images on the social web, ACM Multimedia 2010.
-
What Makes A Post Popular?
Features for prediction User cues
e.g., followers (a users follower count), friends (how many users a user follows), statuses (a users current total post count), user time (a users account creation time), etc.
27
A friend graph:
6
1
-
What Makes A Post Popular? Features for prediction
User cues (topological features) e.g., closeness centrality, the average length of the shortest
path between the node and all other nodes in the graph
28
12
3
45
6 7
-
Closeness Centrality
29
12
3
45
6 7
12
3
45
6 7
0.5
0.67
0.75
0.460.75
0.46 0.46
-
Latent Factor Models
The popularity prediction task is formulated as a matrix completion problem of filling in the missing entries of a partially observed matrix.
30
known popularity
to be estimated
-
Our Observations: Time Matters
31[Ref] http://www.adweek.com/socialtimes/best-time-to-post-social-media/504222
-
Temporal Modeling for Popularity
To incorporate the temporal evolving structures in popularity prediction
32
-
The popularity evolving at multi-granularities with different patterns
33
Challenge 1: Temporal Evolving
Multi-granularities Characteristics of Popularity Dynamics
-
Challenge 2: Data Noise
Popularity patterns are covered in very noisy behavior data or information
Popularity distribution on time series
-
Our Solution#1 [AAAI16]:Incorporating Multi-Scale Temporal Decomposition
35
popularity matrix time scales
Solver: Multiple Update Rule (D.Lee and Sebastian.Seung 1999)
-
Datasets Data Sets
Over 1.8M photos Over 70K users Views, User profile, Photo stream Metadata, Images, Annotations
Settings
36
User-specific Dataset (UsD)Users 400Images 600K
Photo-mix Dataset (PmD)Users 70K Images 1200K
-
Experiments
Metric: Spearman Correlation Time scales:
period, week, month, season period: morning (8:00am-12:00am), lunch time (12:00am-14:00pm),
afternoon (14:00am-17:00pm), dinner time (17:00am- 20:00pm), evening (20:00am-24:00pm) and sleeping (0:00am-8:00am)
37
-
Our Solution#2 [MM16]:A Multi-scale Temporalization (MT) Framework
38
-
Algorithm: Multi-scale Temporalization (MT)
39
Optimization Updating Steps
Optimization Updating Steps
-
Experiments
40
-
Our Solution#3 [IJCAI17]:Deep Temporal Context Networks (DTCN) We address the problem as a sequential prediction task, where the input is
a user-photo sequence (with time order) while the output is the popularity of a future photo (a photo before its publication on social media)
41
-
Experiments
Prediction performances on TPIC17-100K, 200K, and 400K datasets Metric: Spearman Ranking Correlation
42
-
More Influential Factors: Cultures A voting survey of the 2014 TripAdvisor's Top 10 Attractions in Japan by visitors from
different countries shows how much the favorites for attractions can vary among people from different regions, i.e., different cultures.
43[Ref] 2014 TripAdvisors Top 20 Attractions in Japan: http://www.tripadvisor.com/pages/- HotSpotsJapan.html.
-
Foursquare Datasethttps://sites.google.com/site/yangdingqi/home/foursquare-dataset
Individual check-ins data of the more than 10 million users on Foursquare
44
-
A Pilot Study: Understanding Foursquare Venue Popularity in Taiwan
45
Performed by Mr. Mrinal Kanti Baowaly in 2016
-
Taiwan vs. USA Venue Distribution of Top 10 Categories
46
Taiwan
USA
-
More Influential Factors: Personalization What Your Facial Features Say About Your Personality (MM13)
47
personality report
facial image
-
Learning Relevance by Neighbor Voting
48
[Ref] X. Li, C.G.M. Snoek, M. Worring, Learning tag relevance by neighbor voting for social image retrieval, Proc. ACM Intl. Conf. Multimedia Information Retrieval (MIR), 2008.
-
More Influential Factors: Personal Fashion Flavor
49
[Ref] Fashion Analysis: Current Techniques and Future Directions, IEEE Multimedia, 2014.
-
Urban Tribes: Analyzing Group Photos from a Social Perspective [CVPR12]
50
Urban tribe: the term to describe subcultures of people who share common interests and tend to have similar styles of dress, to behave similarly, and to congregate together. (coined by French sociologist Michel Maffesoli in 1985)
Which groups of people would more likely choose to interact socially? (a) and (b) or (a) and (c)?
-
Clothing Fashion Analysis
51
"i-Stylist: Finding the Right Dress Through Your Social Networks," MMM 2017.
"A Framework of Enlarging Face Datasets Used for Makeup Face Analysis," BigMM 2016.
"What are the Fashion Trends in New York?" MM 2014. (Grand Challenge Prize)
"Clothing Genre Classification by Exploiting the Style Elements," MM 2012.
-
Clothing fashion is a reflection of the society of a period The global fashion apparel market today has surpassed
1 trillion US dollars since 2013, and accounts for nearly 2 percent of the world's Gross Domestic Product (GDP)
52
-
Trend Analysis for the Clothing Fashion
OurworkreceivedMultimediaGrandChallengeAwardin2014ACMMultimediaConference.
-
Applications: Fashion is becoming mobile first with apps that help track down must-have clothes, accessories and shoes - theguardian.com
LIKEtoKNOW.it The Netbook
Snap Fashion
The Hunt
-
http://www.fashiontv.com/videos/fashion-weeks
Construct a fashion show dataset
Source:NewYorkFashionWeeks
-
Color Cut
Pattern Head decoration
major elements forfashion style investigation
key factors for discovering fashion trends: coherence (frequently occur within a fashion week) uniqueness (occur much more often in a fashion week than in other fashion weeks)
-
http://www.fashiontv.com/videos/fashion-weeks
Detect the presence of catwalk models over all video frames
...
...
...
...
...
...
http://www.fashiontv.com/videos/fashion-weeks
...
...
...
...
...
...
...
...
...
...
...
...
Identify distinct catwalk models
-
...
...
...
...
...
...
Identify distinct catwalk modelsExtract model location and the full-body image
-
Collect full-body image of catwalk models
Catwalk Models
e.g.NYFWAutumn/Winter2014Positiveset Negativeset
e.g.allcatwalkmodelsatNYFWexceptforAutumn/Winter2014
Divide the collection of full-body images into two sets
DistributionalclusteringtechniqueW.H.Chengetal.,"LearningandRecognitionofOnPremiseSigns(OPSs)fromWeaklyLabeledStreetViewImages,"IEEETran.onImageProcessing(TIP),2014.
-
Query Image
Query Image
Color Analysis Texture Analysis Color + Texture Analysis
Query ImageQuery Image
Color Analysis Texture Analysis Color + Texture Analysis
Query Image Query Image Query Image
Color Analysis Texture Analysis Color + Texture Analysis
Query Image Query Image Query Image
Color Analysis Texture Analysis Color + Texture Analysis
Query Image Query Image Query ImageSpring/Summer
2011Spring/Summer
2013Spring/Summer
2013
Spring/Summer 2011 Spring/Summer 2013 Spring/Summer 2013
-
Predicting Occupation via Human Clothing and Contexts [ICCV11] Diving into the recognition of high-level semantic
categories of human such as occupations
61
-
Recognizing City Identity via Attribute Analysis of Geo-tagged Images [ECCV14] A set of 7 high-level attributes is used to describe the spatial
form of a city (amount of vertical buildings, type of architecture, water coverage, and green space coverage) and its social functionality (transportation network, athletic activity, and social activity).
62
-
From Scene Attributes to City Attributes 102 scene attributes are defined. Each of the city attribute classifier is modeled as an ensemble of SVMs.
63
-
Spatial Analysis of City Attributes The city perception map visualizes the spatial distribution of the 7 city
attributes in different colors and exhibits the visitors and inhabitants own experience and perception of the cities, while it reflects the spatial popularity of places in the city across attributes.
64
-
Attribute-Based City Identity Recognition
65
-
66
Sociological understanding of humans and human interactions is fun but still a long way to go!
-
67
ACM Multimedia 2017http://www.acmmm.org/2017/
Grand ChallengeSocial Media Prediction (SMP):Predicting the Stars of Tomorrow on Social Mediahttps://social-media-prediction.github.io/MM17PredictionChallenge/
Organizers
Wen-Huang Cheng
Academia Sinica
Bo Wu
Chinese Academy of Sciences
Yongdong Zhang
Chinese Academy of Sciences
Tao Mei
Microsoft Research Asis
-
68
-
Yahoo! Datasethttp://webscope.sandbox.yahoo.com/
69
-
YFCC100M This dataset contains 100 million media objects and
explain the rationale behind its creation. This list is compiled from data available on Yahoo! Flickr.
70
Two photos of real world scenes from photographers in the YFCC100M dataset.
-
YFCC100M
Global coverage of a sample of one million photos from the YFCC100M dataset.
71
-
Yelp Dataset Challengehttps://www.yelp.com/dataset_challenge
72
-
Visual Genome Datasethttps://visualgenome.org/
73
-
Instagram Datasethttp://www.emilio.ferrara.name/datasets/
74
-
ICWSM-16 DatasetInternational Conference on Web and Social Media
http://www.icwsm.org/2016/datasets/datasets/
75
-
76
General Chairs Program Chairs
Wan-Chi SiuHong Kong Polytechnic University
Chia-Wen LinNational Tsinghua University
Wen-Huang ChengAcademia Sinica
Gene CheungNational Institute of Informatics
vcip2018.org
-
Lets exchange ideas!
77
Wen-Huang Cheng
wenhuangcheng