recsysfr: criteo presentation

RecSys FR: 4th Session

6th October 2016

Etienne Sanson – manager R&D Engine

About Criteo

1

3 | Copyright © 2016 Criteo

Our mission

TARGET THE RIGHT USER

AT THE RIGHT TIME

WITH THE RIGHT MESSAGE


Key Figures

16 000 PUBLISHERS

90% RETENTION RATE2

+130COUNTRIES

LISTED ON THE

NASDAQ SINCE

OCTOBER 2013

R&D REPRESENTS 21% OF THE WORKFORCE

2000EMPLOYEES

21 BILLIONS $3

11 000 ADVERTISERS

1.19 bn€1

31OFFICES

1: REVENUE IN 20152: ANNUAL RATE 2015

3: $ OF TURNOVER GENERATED TO OUR CLIENTS - TURNOVER POST-CLICK WW FROM JANUARY TO DECEMBER 2015


Revenue Growth

2009 2010 2011 2012 2013

22M$86M$

199M$

349M$

589M$

988M$

2014

1,3MM$

2015


GENERAL CONCEPT

Users visit an advertiser’s website

1

Criteo identifies the users (via cookies)

2

Users leave the advertiser’s website& browse publisher on the Internet

3

Criteo identifies users on these pages(via cookie)

4

Criteo displays an advertising banner, personalized for

each user

5

Click through directlyto the advertiser’s

page

6

@

Retargeting principles


Infrastructure Key Figures

Sunnyvale2 PoP

500 kVA1 559 Servers

New York2 PoP


Hong Kong2 PoP

472 kVA2185 Servers

Paris4 Pop

1 800 kVA3 625 Servers

Amsterdam2 PoP

+2 500 kVA3 609 Servers

Tokyo2 PoP


Shanghai1 PoP

200 kVA931 Servers

World Wide15 PoP

6,8 MVA17 098 Servers

> 55Gbps+ 2.5M req/s

Hosting Global Partners :

About ML@Criteo

2


Our challenges – Product recommendation

• Select the best ~10 products to show to a user

• >1B users• Product catalog contains ~1M items, up to 1B• Time constraints: 20ms

• Combination of offline/online processing steps

• CF• Product embeddings (word2vec -> prod2vec)• CNN for image features

What products should we recommend?


Our challenges - Bidding

How much should we bid for this display?

What is the best campaign to display?

My company

BUY! BUY! BUY!

BUY!

• Select the best campaign to display and evaluate its value in a few ms

• Large scale regression models• >1B daily displays (but few positive examples!)• >1M parameters

• Distributed optimization (SGD, L-BFGS)• Feature Engineering• Transfer learning, FFM, Policy learning• Marketplace, game theory, auction theory


Our challenges – X-device

• Build a huge graph (Billions of nodes/edges):• Nodes = devices• Edge = the 2 devices belong to the same user

• How to connect 2 devices?• How to know the ground truth?• How to keep it stable?

• At scale & taking care about privacy

Who is the user behind the device?


Our challenges – Testing

• We test everything!

• Offline tests / AB Tests• Infrastructure to perform large-scale tests

• >100K offline tests / year• >1K AB Tests / year• Dedicated teams

• Technical / Business Metrics• Randomization• Counterfactual evaluation

Thank you!

…and we’re hiring!