diversity and novelty for recommendation system
DESCRIPTION
A simple survey of Diversity and novelty metrics for recommender systemsTRANSCRIPT
A simple survey of Diversity and A simple survey of Diversity and novelty metrics for recommender novelty metrics for recommender systemssystems
Reporter:孙建凯
2012.07.11
Copyright 2012 by IRLAB@SDU
Move beyond accuracy metricsMove beyond accuracy metrics
while the majority of algorithms proposed in recommender systems literature have focused on improving recommendation accuracy
other important aspects of recommendation quality, such as the diversity of recommendations, have often been overlooked.
The recommendations that are most accurate according to the standard metrics are sometimes not the recommendations that are most useful to users[1]
2
Copyright 2012 by IRLAB@SDU
Diversity and Novelty
Accurate is not always good: How Accuracy Metrics have hurt Recommender Systems
GroupLensResearch,CHI'06
Copyright 2012 by IRLAB@SDU
Accuracy does not tell the whole story
Copyright 2012 by IRLAB@SDU
Diversity
Individual Diversity Aggregate Diversity
Copyright 2012 by IRLAB@SDU
Individual Diversity
Diversity Difficulty[3]
Average dissimilarity between all pairs of items recommended to a given user(intra-list similarity) [2,4]
Copyright 2012 by IRLAB@SDU
Diversity Difficulty
What We Talk About When We Talk About Diversity [DDR’12 Northeastern University USA]
Like query difficulty in IR
For a specific query and corpus, query difficulty is a measure of how successful the average search engine should be at ad-hoc retrieval.
Copyright 2012 by IRLAB@SDU
Diversity Difficulty
Diversity Difficulty is defined with respect to a query and a corpus.
Describes diversity-the number of subtopics which are covered by a list;
Describes novelty-which is inversely proportional to the number of times a list repeats a subtopic
Copyright 2012 by IRLAB@SDU
Finding needles in the haystack
Imagine a query with 10 subtopics ,1000 documents relevant to only the first subtopic, and each of the remaining subtopics covered by a single, unique document.
On the other hand ,if there are large numbers of documents relevant to multiple subtopics, it would be easy to produce a diversity list.
Copyright 2012 by IRLAB@SDU
Diversity Difficulty function
The maximum amount of diversity achievable by any ranked list-dmax
The ease with a system can produce a diverse ranked list.-dmean
Harmonic function
Copyright 2012 by IRLAB@SDU
Examples
Copyright 2012 by IRLAB@SDU
Improving Recommendation Lists Through Topic Diversification
Introduce the intra-list similarity metric to access the topic diversification of recommendation lists and the topic diversification approach for decreasing the intra-list similarity
Average dissimilarity between all pairs of items recommended to a given user
Copyright 2012 by IRLAB@SDU
Intra-list Similarity
Copyright 2012 by IRLAB@SDU
Taxonomy-based similarity Metrics
Instantiate c with their metric for taxonomy-driven filtering.[5]
Copyright 2012 by IRLAB@SDU
Topic Diversification Algorithm
Algorithm A brief textual sketch
Experiments
precision diversity
Copyright 2012 by IRLAB@SDU
Aggregate Diversity
improving recommendation Diversity using ranking-based techniques[IEEE transaction’12]
Use the total number of distinct items recommended across all users as an aggregate diversity measure, define as follows:
Copyright 2012 by IRLAB@SDU
General overview of ranking-based approaches for improving diversity
Copyright 2012 by IRLAB@SDU
Re-Ranking Approach
Copyright 2012 by IRLAB@SDU
Other Re-ranking Approach
Copyright 2012 by IRLAB@SDU
Other Re-ranking Approach
Copyright 2012 by IRLAB@SDU
Other Re-ranking Approach
Copyright 2012 by IRLAB@SDU
Other Re-ranking Approach
Copyright 2012 by IRLAB@SDU
Other Re-ranking Approach
Copyright 2012 by IRLAB@SDU
Other Re-ranking Approach
Copyright 2012 by IRLAB@SDU
Combining Ranking Approaches
Many possible ways to combine several ranking functions
In this paper , linear combination
Open issue: letor ? Neural network?
Copyright 2012 by IRLAB@SDU
Entropy
A study of Heterogeneity in Recommendations for a social Music Service[6]
Copyright 2012 by IRLAB@SDU
Open issue:probability
Copyright 2012 by IRLAB@SDU
Entropy
Aggregate Entropy:
Item popularity between lists?
Individual Entropy:
subtopic popularity?
Copyright 2012 by IRLAB@SDU
Bipartite network
Bipartite network projection and personal recommendation[Tao Zhou, Physical Review]
Solving the apparent diversity-accuracy dilemma of recommender systems[Tao Zhou]
Copyright 2012 by IRLAB@SDU
Illustration of resource-allocation process in bipartite network
Copyright 2012 by IRLAB@SDU
Solving the apparent diversity-accuracy dilemma
heats probs
Copyright 2012 by IRLAB@SDU
Hybrid Methods
weight hybrid
Copyright 2012 by IRLAB@SDU
Diversity Measure
Copyright 2012 by IRLAB@SDU
Surprisal/novelty
Copyright 2012 by IRLAB@SDU
Results-why better?
Copyright 2012 by IRLAB@SDU
Surprise me
Tangent: A novel, ‘surprise me’, recommendation algorithm [kdd’09]
Copyright 2012 by IRLAB@SDU
Framework of Tangent Algorithm
Suggest items which are not only relevant to user preference but also have a large connectivity to other groups.
Consisting three parts as follows:
1 Calculate relevance score(RS) for each node
2 Calculate bridging score(BRS) for each node
3 Compute the Tangent score by somehow merging two criteria above
Copyright 2012 by IRLAB@SDU
Case study
Copyright 2012 by IRLAB@SDU
Case study
Copyright 2012 by IRLAB@SDU
Call for papers
September 20, 2012
Copyright 2012 by IRLAB@SDU
Reference
1. Accurate is not always good: How Accuracy Metrics have hurt Recommender Systems
2.improving recommendation Diversity using ranking-based techniques
3. What We Talk About When We Talk About Diversity
4. Improving Recommendation Lists Through Topic Diversification
5. Taxonomy-driven computation of product recommendations
Copyright 2012 by IRLAB@SDU
Reference
6. A study of Heterogeneity in Recommendations for a social Music Service
7. Bipartite network projection and personal recommendation
8.Solving the apparent diversity-accuracy dilemma of recommender systems
9. Tangent: A novel, ‘surprise me’, recommendation algorithm
Copyright 2012 by IRLAB@SDU
thanks