2009fs-01

8/3/2019 2009FS-01

1/32

Ranking and Suggesting Popular Items

IDB & IDS Lab. Seminar

Fall 2009

Minsuk [email protected]

September 18th, 2009

M. Vojnovic, J. Cruise, D. Gunawardena, & P. Marbach

IEEE Trans. on Knowledge & Data Eng. (TKDE)Vol. 21, No. 8, August 2009

Center for E-Business TechnologySeoul National UniversitySeoul, Korea Intelligent Database Systems Lab.

8/3/2019 2009FS-01

2/32

Copyright 2009 by CEBT

Abstract


Items are suggested to users to aid tasks such as browsing of the content.

Users tend to select suggested items more frequently (popularity bias).

Propose algorithms for suggesting popular items,

which enables to learn users true preference over items.

2

Suggestion set

8/3/2019 2009FS-01

3/32

Contents

Introduction

Problem Formulation

Algorithms

Additional Material

Analytical & Numerical Results

Related Work

Conclusion

3

8/3/2019 2009FS-01

4/32


Introduction

Item Suggestion

Items are suggested to users to aid tasks

such as browsing or tagging of the content.

Items could be search query keywords, documents, tags, etc.

4

8/3/2019 2009FS-01

5/32


Introduction

Motivation

Learning of item popularity is complicated

Users tend to select suggested items more frequently.

least effort, bandwagon effect

Such popularity bias makes problem.

5

Just Click from Suggestion Set?

Think..., and type new one?

8/3/2019 2009FS-01

6/32


Introduction

Motivating Example

6

8/3/2019 2009FS-01

7/32


Introduction

Motivating Example

7

8/3/2019 2009FS-01

8/32


Introduction

Goal

Propose algorithms and analyze performance for suggesting popular items

A way that enables to learn users true preference over items

8

8/3/2019 2009FS-01

9/32

Contents

Introduction

Problem Formulation

Algorithms

Additional Material


Related Work

Conclusion

9

8/3/2019 2009FS-01

10/32


Problem Formulation

Ranking and Suggesting Items

Problem

Users select items from a given set C

ris Users true preference over the set of items C

rcalled true popularity rank scores

ri is the portion of users that would select item i if suggestions were not made

Assumption

ri is strictly positive

Items are enumerated

r is normalized such that it is a probability distribution.

10

ri 0.45 0.21 0.13 0.10 0.06 0.05

8/3/2019 2009FS-01

11/32


Problem Formulation

Algorithms

Ranking rule

Specifies how to update the ranking scores of items

Suggestion rule

Specifies what subset of items to suggest to a user

Assumption

size of suggestion set is fixed

Do not account for suggestion order

interested in small suggestion sets

allow for randomized order

11

8/3/2019 2009FS-01

12/32


Problem Formulation

Design objective

Learn the true popularity ranking of items

For items, i and j, rirj implies i (t)j(t), for sufficiently large time t

r is the true popularity rank scores

is the ranking scores generated by ranking rule algorithm

Suggest true popular items and identify quickly the true popular items

Evaluation metric

Precision of a set of items

12

8/3/2019 2009FS-01

13/32


entire item set

Problem Formulation

Users Choice Model

How users choose items when he is presented a set S of suggested items

User selects an item from the entire set of items by sampling,

using the true item popularity distribution r.

Otherwise, user does the same but confines his choice to items in the suggest set.

13

Suggestion set

8/3/2019 2009FS-01

14/32

Contents

Introduction

Problem Formulation Algorithms

A Nave Algorithm

Ranking rules

Suggestion rules

Additional Material Analytical & Numerical Results

Related Work

Conclusion

14

8/3/2019 2009FS-01

15/32


A B C D E F

Algorithms

A Nave Algorithm

TOP (Top popular)

Simple algorithm (baseline)

Rank score of an item equals number of selections of this item.

15

Vi 3 2 1 2 1 1

8/3/2019 2009FS-01

16/32


Top Popular

Algorithms

Proposed algorithms

16

nave

not biased

ranking suggesting

Rank rule 2

Rank rule 1

PROP

M2S

FM2S

top-N rank score(baseline)

8/3/2019 2009FS-01

17/32


A B C D E F

pi

count

Algorithms

Ranking rules

Rank rule 1

Simple ranking rule

Rank score for item i increases whenever a user selects this item.

17

i 0.33 0.17 0.17 0.08 0.08 0.08

count 3 2 2 1 1 1

8/3/2019 2009FS-01

18/32


Algorithms

Ranking rules

Rank rule 2

Rank rule 1 may fail to discover true popularity order.

Here, rank score updated only for an item that was not suggested.

Slow rate of convergence

18

A B C D E F

pi

count

i 0.33 0.17 0.11 0.03 0.04 0.04

count 12 13 1 1 1 2

8/3/2019 2009FS-01

19/32


Algorithms

Suggestion rules

PROP (Frequency Proportional)

randomized algorithm

Suggestion set is sampled with probability proportional to current rank score.

More robust to imitation than TOP

19

A B C D E F

Suggestion set

8/3/2019 2009FS-01

20/32


Algorithms

Suggestion rules

M2S (Move-to-set)

Suggest the last used item

Suggestion set updated only when a user selects an item that is not in S

Random iterative update rule of suggestion set

computationally very simple

20

Suggestion set

8/3/2019 2009FS-01

21/32


Algorithms

Suggestion rules

FM2S (Frequency move-to-set)

can go to suggestion set only if sufficiently popular, w.r.t. true popularity

compared to M2S (previous page)

Not update counter for an item that were in suggested set

different from TOP (first simple alg.)

21

8/3/2019 2009FS-01

22/32


Suggestion set

A B C D E F G H

Algorithms

Suggestion rules

FM2S (Frequency move-to-set)

can go to suggestion set only if sufficiently popular, w.r.t. true popularity

Not update counter for an item that were in suggested set

22

+3 +2A B C D E F G H

Wi 7 7 6 5 5 4 2 1

8/3/2019 2009FS-01

23/32

Contents

Introduction

Problem Formulation

Algorithms

Additional Material


Related Work

Conclusion

23

8/3/2019 2009FS-01

24/32


Additional Material

Experiment done by me

TOP vs. FM2S

TOP fail to catch true distribution

100 times of item selection sampled from true preference distribution

when imitation probability (probability for selecting from suggestion set) is 0.5

24

true 25 20 15 11 10 7 6 6

count 23 86 30 13 12 9 24 3

FM2S 12 12 13 11 9 9 12 3

8/3/2019 2009FS-01

25/32

Contents

Introduction

Related Work

Problem Formulation

Algorithms

Additional Material

Analytical & Numerical Results Analytical Results

Numerical Results

Conclusion

25

8/3/2019 2009FS-01

26/32


Analytical Results

Goal

present analytical results on the ranking and suggesting algorithms

Results

some Theorems with proof

Top Popular

There exists a threshold on the imitation probability belowwhich the algorithm guarantees to learn true popularity rank.

The threshold is a function of suggestion set size and true popularity rank scores.

Frequency Proportional

Move-to-Set

Frequency Move-to-Set

Show that this algorithm tends to suggest only a subset of

sufficiently true popular items

26

8/3/2019 2009FS-01

27/32


Numerical Results

Goal

Evaluate performance of algorithms

using samples of real-world distributions for item popularity scores

Data set

Tagging histories for bookmarks from del.icio.us

Study

Learning the true popularity ranking

compute threshold imitation probability for each bookmark

Undesired lockdown may happen at small level of imitation (about 0.1)

Precision of suggestions evaluate mean precision of suggestions made by algorithms PROP, M2S, FM2S

Mean precision under FM2S is the best

27

8/3/2019 2009FS-01

28/32

Contents

Introduction

Problem Formulation

Algorithms

Additional Material


Related Work

Conclusion

28

8/3/2019 2009FS-01

29/32


Related Work

Related Area

Recommendation systems

learn which items preferred based on users selection

We consider a system with feedback

Voting systems

an instance of approval voting

Feedback may bias voting results

bandwagon and underdog effects induced by pre-election polls

Use click-through data for Web search

search engine result sets lock down to a set of popular URLs

Our problem is different because user can select items that is not presented. Social Tagging

The effect of the tag suggestions

Tags applied by users are affected by tag suggestions

29

8/3/2019 2009FS-01

30/32

Contents

Introduction

Related Work

Problem Formulation

Algorithms


Conclusion

30

8/3/2019 2009FS-01

31/32


Suggestion set

entire item set

Conclusion


propose randomized algorithms for ranking and suggesting popular items

designed to account for popularity bias.

M2S and FM2S learn true popularity ranking that are lightweight

self-tuning in that they do not require any special configuration parameters

FM2S confines to displaying only sufficiently popular items

31

nave

notbiased

ranking suggesting

Rank rule2

Rank rule1

FM2S

top-N rank score(baseline)

Top Popular

8/3/2019 2009FS-01

32/32

32

Thank you~

2009fs-01

Documents