expressiveness in mechanisms and its relation to efficiency: our experience from $40 billion of...

Expressiveness in mechanisms and its relation to efficiency:

Our experience from $40 billion of combinatorial multi-attribute auctions, and recent theory

Tuomas Sandholm

Founder, Chairman, Chief ScientistCombineNet, Inc.

ProfessorComputer Science Department

Carnegie Mellon University

Outline

• Practical experiences with expressiveness

• Domain-independent measure of expressiveness– Theory on how it relates to efficiency

• Application of the theory to sponsored search

• Expressive ad (e.g., banner) auction that spans time

Sourcing before 2000

Structured, transparent

Simultaneous negotiation with all suppliers

Global competition

Basic reverse auction

Unstructured, nontransparent

Sequential => difficult, suboptimal decisions

1-to-1 => lack of competition

Expressive => win-win

Implementable solution

Manual negotiation

ConsPros

Bidding on predetermined lots is not expressive => ~ 0-sum game

Lotting effort

Small suppliers can’t compete

Unimplementable solution

Bidding complexity & exposure

Expressive commerce

• Expressive bidding

• Expressive allocation evaluation

Expressive bidding• Package bids of different forms

• Conditional discount offers of different forms (general trigger conditions, effects, combinations & sequencing)

• Discount schedules of different forms

• Side constraints, e.g. capacity constraints

• Multi-attribute bidding – alternates

• Detailed cost structures

• All of these are used in conjunction

– Don’t have to be used by all

Benefits of expressive bidding

Pareto improvement in allocation1. Finer-grained matching of supply and demand (e.g. less empty driving)

2. Exposure problems removed => better allocation & lower cost

3. Capacity constraints => suppliers can bid on everything

4. No need to pre-bundle => better bundling & less effort

5. Fosters creativity and innovation by suppliers

6. Collaborative bids

=> lower prices and better supplier relationships

Academic bidding languages unusable (in this application)

• OR [S. 99]• XOR [S. 99]• OR-of-XORs [S. 99]• XOR-of-ORs [Nisan 00]• OR* [Fujishima et al. 99, Nisan 00]• Recursive logical bidding languages

[Boutilier & Hoos 01]

Fully expressive

Expressive allocation evaluation

• Side constraints– Counting constraints– Cost constraints– Unit constraints– Mixture constraints– …

• Expressions of how to evaluate bidder and bid attributes

Example of expressive allocation evaluation

Benefits of expressive allocation evaluation

1. Operational & legal constraints captured => implementable allocation

2. Can honor prior contractual obligations

3. Speed to contract: months weeks– $ savings begin to accrue earlier

– Effort savings

Clearing (aka. winner determination) problem• Allocate (& define) the business– so as to minimize cost (adjusted for buyer’s preferences) – subject to satisfying all constraints

• Even simple subclass NP-complete & inapproximable [S., Suri, Gilpin & Levine AAMAS-02]

• We solve problems ~100x bigger than competitors, on all dimensions: • > 2,600,000 bids • > 160,000 items (multiple units of each)• > 300,000 side constraints• > 1,000 suppliers

• Avg 20 sec, median 1 sec, some instances take days• Speed & expressiveness: huge competitive advantage

CombineNet events so far• > 500 procurement events– $2 million - $1.6 billion– The most expressive auctions ever conducted

• Total transaction volume > $40 billion• Created 12.6% savings for customers– Constrained; Unconstrained was 15.4%

• Suppliers also benefited– Positive feedback (win-win, expression of efficiencies,

differentiation, creativity)– Un-boycotting– They recommend use of CombineNet to other buyers

Applied to many areasChemicalsAromaticsSolventsCylinder GassesColorants

PackagingCans & EndsCorrugated BoxesCorrugated DisplaysFlexible FilmFolding CartonsLabelsPlastic Caps/ClosuresShrink/Stretch Film

Ingredients/Raw Mat.Sugars/SweetenersMeat/Protein

TransportationAirfreightOcean FreightDrayTruckloadLess-than-truckload (LTL)BulkSmall ParcelIntermodal3PLs

Industrial Parts/MaterialsBulk ElectricFastenersFiltersLeased EquipmentMROPipes/Valves/Fittings/GaugesPumpsSafety SuppliesSteel

MarketingMedia buyCorrugate DisplaysPrinted Materials Promotional Items

TechnologySecurity CamerasComputers

ServicesPre-pressTemporary LaborShuttling/TowingWarehousing

MedicalPharmaceuticalsMedical/surgical supplies

MiscellaneousOffice Supplies

Broader trend toward expressiveness

Amazon.c & New Egg o offer bundles of items (ca. 2000)

Facebook increases expressiveness of privacy control (2006)

“…we did a bad job of explaining what the new features were and an even worse job of giving you control of

them…. This is the same reason we have built extensive privacy settings — to give you even more control over who you share your information with.”

CD+Tunes adds option for users to rent movies (2007)

Checked baggage

Airlines charge extra for baggage, food & choice seats (2008)

Prediction/insurance markets becoming more expressive

Is more expressiveness always better?

• Not always for revenue!

Expressive mechanism: vi( )? vi ( )? vi ( )?

An inexpressive mechanism: vi ( )?

Is more expressiveness always betterfor efficiency?

• And what is expressiveness, really?

[Benisch, Sadeh & S. AAAI-08]

What makes a mechanism expressive?A straw man notion

$5 $2

Expression space 2

Item bid auction

$5 $2 $6

Expression space 3

Combinatorial auction

What makes a mechanism expressive?

Prop: Dimensionality of expression space does not suffice

Expression space 1

a b

Mapping

Proof intuition [based on work of Georg Cantor, 1890] :

Expression space 3

Work on informational complexity in mechanisms [Hurwicz, Mount, Reiter 1970s…] puts technical restrictions that preclude such mappings

Our notion: Expressive mechanisms allow agents more impact on outcome

An agent’s impact is a measure of the outcomes it can choose between by altering only its own expression

$X

$Y

$6

$4 AB D

C $X $Y $6$4

Uncertainty introduces the need for greater impact

$X $Y

$6$4

$X

$Y

$4

$6

A,A A,C C,C

C,D

D,DB,D

A,DA,B

B,B

Uncertainty introduces the need for greater impact…

$X $Y

$3$7

$X

$Y

$3

$7A,A A,C C,C

C,D

D,DB,D

A,DA,B

B,B

Uncertainty introduces the need for greater impact…

$X $Y

$6$4 $3$7

$3

$4

$6

$7

$X

$YA,A A,C C,C

C,D

D,DB,D

A,DA,B

B,B

•10 outcome pairs but only 9 regions•In this example the impact vector B,C can’t be

expressed

Expressive mechanisms

$6$4 $3$7

• Our measure of expressiveness for one agent (semi-shattering): how many combinations of outcomes can he choose among• Not just for combinatorial allocation problems because outcomes can be anything• Captures multi-attribute considerations as well

$X $Y $Z

$X

$Y

D,DB,D

A,B

B,B

B,C

A,A ExtraRegion

$X

$YA,A A,C C,C

C,D

D,DB,D

A,DA,B

B,B

Z=0

Some Z>0

$X

$Y

$Z

• In combinatorial auction all 10 pairs can be expressed

$X

$YA,A A,C C,C

C,D

D,DB,D

A,DA,B

B,B

An upper bound on a mechanism’s best-case efficiency

• We study a mechanism’s efficiency when agents cooperate

• It bounds the efficiency of any equilibrium

• It allows us to avoid computing equilibrium strategies

• It allows us to restrict our analysis to pure strategies only

? ?

Theorem: the upper bound on efficiency for an optimal mechanism increases strictly monotonically as more expressiveness (# of expressible impact vectors) is allowed (until full efficiency is reached)

Proof intuition: induction on the number of expressible impact vectors; each time this is increased at least one more efficient outcome is allowed

Theorem: the upper bound on efficiency for an optimal mechanism can increase arbitrarily when any increase in expressiveness (# of expressible impact vectors) is allowed

Proof intuition: construct preference distributions that ensure at least one type makes each combination of outcomes arbitrarily more efficient than any others

The bound can always be met

Theorem: for any outcome function, there exists at least one payment function that yields a mechanism that achieves the bound's efficiency in Bayes-Nash equilibrium Proof intuition: if agents are charged their expected imposed externality (i.e., the inconvenience that they cause to other agents in the potentially inexpressive mechanism), then making expressions that maximize social welfare is an optimal strategy for each agent given that the others do so as well

Application to sponsored search

[Benisch, Sadeh & S. Ad Auctions Workshop 2008]

0% 20% 40% 60% 80% 100%

-$0.15

-$0.05

$0.05

$0.15

$0.25Prototypical

value advertiser Prototypicalbrand advertiser

Bidd

er u

tility

Rankpercentile

Heterogeneous bidder preferences

Mechanisms we compared

Inexpressivemechanism

$4

Rank

4321

Premiummechanism

$5

$44321

Fully expressive mechanism

$5$4$3$24

321

Google, Yahoo!,

Microsoft, …Our proposal

Expressiveness

50%

60%

70%

80%

90%

100%

Best

-cas

e ex

pect

ed e

ffici

ency

Inexpressivemechanism

Premiummechanism

Fully expressivemechanism

[Boutilier, Parkes, S. & Walsh AAAI-08]

Expressive ad (e.g., banner) auctions that span time,

and model-based online optimization for clearing

Prior expressiveness

• Typical expressiveness in existing ad auctions– Acceptable attributes– Per-unit bidding (per-impression/per-clickthrough (CT))– Budgets– Single-period expressiveness (e.g., 1 day)

• Most prior research assumes this level of expressiveness

Campaign-level expressiveness• Advertising campaigns express preferences over a sequence of allocations

– Minimum targets: pay only if 100K impressions in a week– Tiered preferences: $0.20 per impression up to 30K, $0.50 per impression for more– Temporal sequencing: at least 20K impressions per day for 14 days– Substitution: either NYT ($0.90) or CNN ($0.50) but not both– Smoothness: impressions vary by no more than 20% daily– Long-term budget: spend no more that $250k in a month– Exclusivity

• Additional forms of expressiveness– Advertiser’s choice of impression/CT/conversion pricing (or combination)– Target audience (e.g., demographics) rather than indirectly via web site properties

• Bidder 1: bids $1 on A, $0.50 on B, budget $50k• Bidder 2: bids $0.50 on A, budget $20k• Traditional first-price auction: $52.3k revenue

Value of optimization under sequential expressiveness

Supply of A

Supply of B

t0 t1 t2

50k 0

. . .

. . .

10k 10k

Bidder 2: 4.55kBidder 1: 45.45k

Bidder 1: 9.09k

• Bidder 1: bids $1 on A, $0.50 on B, budget $50k• Bidder 2: bids $0.50 on A, budget $20k• Optimal allocation: $70k revenue

Supply of A

Supply of B

t0 t1 t2

50k 0

. . .

. . .

10k 10k

Bidder 1: 10k

Bidder 1: 80k

Bidder 2: 40k

Value of optimization under sequential expressiveness

Stochastic optimization problem

• Advertising channels C• Supply distribution of advertising channels PS

• Set of campaigns B• Spot market demand distribution PD

• Time horizon T• Can be modeled as Markov Decision Process (MDP)– But how do we make it scale?

Scalable optimization with real-time response

• Huge number of possible events => infeasible to compute full policy contingent on all future states

• Cannot reoptimize policy in real time at every event• Optimize-and-dispatch architecture [Parkes and S., 2005]

– Periodically compute policies with limited contingencies (e.g., stop dispatching when budget reached)

– Dispatch in real time

• Policy form: xti,j - fraction of channel i allocated to campaign j at time t

• Optimize over coarse time periods (e.g., minutes, hours)– Tradeoff between optimization speed and optimality– Finer-grained in near-term, coarse-grained in long-term

Channels• A channel is an aggregation of properties (web pages or spots on them)• Constructed automatically based on campaigns• Lossless aggregation: two web pages are in the same channel if

indistinguishable from the point of view of bids• Example:

– Bid 1: NY Times (NYT)– Bid 2: Medical article (Med)– Channels: (NYT ∧ Med), (NYT ∧ ¬Med), (¬ NYT ∧ Med)– Non-NYT pages grouped together, non-Med pages grouped together

• We can also perform lossy abstraction to avoid exponential blowup

Algorithms for stochastic problem

• Infeasible to solve the MDP– Huge state space – cross product of individual

campaign states– High-dimensional continuous action space

• Our approaches:– Deterministic optimization– Online stochastic optimization

Deterministic optimization• Replace uncertain channel supply with expectations• Formulate the problem as a mixed-integer program (MIP)• Solving a MIP is much faster than an MDP• Our winner determination algorithms can solve very large problems [S. 2007]

• Solutions may be far from optimal if supply distributions have high variance– Does not adequately account for risk– Can be mitigated by periodic reoptimization

Sample-based online stochastic optimization [van Hentenryck & Bent 06]

• Compute only next action, rather than entire policy– Informed by what we might do in the future– Recompute at each time period

• Sample-based– Solve w.r.t. samples from distributions

• Extremely effective when good deterministic algorithms exist

• Requires that domain uncertainty is exogenous– Distribution of future events doesn’t depend on decisions– Roughly true for advertising: allocation of ads should

have little effect on supply of channel

REGRETS algorithm [Bent & van Hentenryck 04]

Choose xt,i thatmaximizes f(xt,i)

Ttt xxx 21

22 ,,,

Ttt xxx 11

11 ,,,

Sample λ1

Optimal solution

.

.

.

Sample λ2 Optimal solution

Sample λK

Optimal solution

λ1t+1λ1

t λ1t+2 λ1

T...λ1t+3 λ1

t+4

λ2t+1λ2

t λ2t+2 λ2

T...λ2t+3 λ2

t+4

λKt+1λK

t λKt+2 λK

T...λKt+3 λK

t+4

TK

tK

tK xxx ,,, 1

Time tAction Value

xt,1 f(xt,1)

xt,2 f(xt,2)

xt,n f(xt,n)

xt,3 f(xt,3)

.

.

.

.

.

.

Ttt xxx 21

22 ,,,

Ttt xxx 11

11 ,,,

Sample λ1

Optimal solution

.

.

.

Sample λ2 Optimal solution

Sample λK

Optimal solution

λ1t+1λ1

t λ1t+2 λ1

T...λ1t+3 λ1

t+4

λ2t+1λ2

t λ2t+2 λ2

T...λ2t+3 λ2

t+4

λKt+1λK

t λKt+2 λK

T...λKt+3 λK

t+4

TK

tK

tK xxx ,,, 1

)( txf

Lower bound on Q-valuesfor action xt at time t

)(

,,,

2

21

2

tt

Ttt

xQ

xxx

)(

,,,

1

11

1

tt

Ttt

xQ

xxx

)(

,,, 1

ttK

TK

tK

t

xQ

xxx

.

.

.

REGRETS algorithm [Bent & van Hentenryck 04]

REGRETS doesn’t apply to ad auctions

• Requires set of possible first-period decisions to be small• Our dispatch policies are continuous• Even a discretization of our continuous decision space

would be huge: dimensionality = |C||B||Discretization|

Our extension of REGRETS to continuous action spaces

Ttt xxx 21

22 ,,,

Ttt xxx 11

11 ,,,

Sample λ1

Optimal solution from MIP:

.

.

.

Sample λ2 Optimal solution from MIP:

Sample λK

Optimal solution from MIP:

λ1t+1λ1

t λ1t+2 λ1

T...λ1t+3 λ1

t+4

λ2t+1λ2

t λ2t+2 λ2

T...λ2t+3 λ2

t+4

λKt+1λK

t λKt+2 λK

T...λKt+3 λK

t+4

TK

tK

tK xxx ,,, 1

)(

,,,

2

21

2

tt

Ttt

xQ

xxx

)(

,,,

1

11

1

tt

Ttt

xQ

xxx

)(

,,, 1

ttK

TK

tK

t

xQ

xxx

Kk

ttk

xxQ

Kt)(

1max

Combining MIP

.

.

.

Revenue: Flat bids

Method Unimodal supply Bimodal supply

Bid-all 25,687 ± 436 14,004 ± 141

Myopic 30,256 ± 437 15,890 ± 175

Deterministic

42,365 ± 581 22,385 ± 227

Stochastic 42,237 ± 581 22,774 ± 238

Revenue: Bonus bids

Method Unimodal supply Bimodal supply

Deterministic 100,266 ± 3,555 55,901 ± 1,887

Stochastic 149,423 ± 3,204 65,065 ± 2,356

Conclusions & future research• Expressive mechanisms are practical & provide huge benefits• Needed to develop natural concise expressiveness forms

– Prior academic bidding languages not usable• For efficiency, can add any expressiveness forms

– Will help & can help an arbitrary amount; Bound can be met in BNE• Uncertainty about others => need more expressiveness

– Unlike in work solely on dominant-strategy mechanisms [Ronen 01], [Holzman et al. 04], [Blumrosen & Feldman 06]

• Sponsored search– GSP seems to run at a large inefficiency– Most of it fixable by our “premium mechanism”

• Expressive (banner) ad auctions that span time– Optimize-and-dispatch framework– Channel aggregation– Deterministic optimization with re-optimization– Online sample-based optimization – extended to continuous action space– Optimization provides significant benefits, even with no added expressiveness– Stochastic especially beneficial with non-linear preferences

• Most helpful expressiveness forms for other apps?– Agents’ preferences as input, our methodology can be used to evaluate different mechanisms– Psychological burden: expressing more vs. expressing strategically (e.g., chopsticks)