slides david kahle

8/12/2019 Slides David Kahle

1/97

Introducing Algebraic Statistics

David Kahle

Assistant Professor

Department of Statistical Science

June 3, 2013

Introducing Algebraic StatisticsDavid Kahle


2/97

Minimum Distance Estimation in Categorical CI Models2

Outline

u n i v e r s i t y

BAYLOR


It is true that few unscientific people have this particular type of religious

experience. Our poets do not write about it; our artists do not try to

portray this remarkable thing. I don't know why. Is no one inspired by our

present picture of the universe? This value of science remains unsung by

singers: you are reduced to hearing not a song or poem, but an evening

lecture about it. This is not yet a scientific age.

Richard Feynman, The Value of Science


3/97

u n i v e r s i t y

BAYLOR


Whats algebraic statistics?


4/97

u n i v e r s i t y

BAYLOR



Algebraic statistics is concerned with the development of techniques inalgebraic geometry, commutative algebra, and combinatorics, to addressproblems in statistics and its applications.

Drton, Sturmfels, and Sullivant

Lectures on Algebraic Statistics, 2009


5/97

u n i v e r s i t y

BAYLOR






Disclaimer :Im not an algebraist!


6/97

u n i v e r s i t y

BAYLOR







Great text :Cox, Little, and OSheas

Ideals, Varieties, and Algorithms


7/97

u n i v e r s i t y

BAYLOR






Areas of application

Multiway contingency tables

Graphical models Factor analysis

Structural equations models

Statistical disclosure limitation

Evolutionary biology

Causal models Mixture models

Bayesian integrals



8/97

u n i v e r s i t y

BAYLOR


My multivariatecalculus nightmare :

5. Sketch x2 y2 z2= 1/2.


9/97

u n i v e r s i t y

BAYLOR



5. Sketch x2 y2 z2= 1/2.

Lets ask !!


10/97

u n i v e r s i t y

BAYLOR



5. Sketch x2 y2 z2= 1/2.

Lets ask !!

ContourPlot3D[

x^2 - y^2 - z^2 == 1/2,

{x, -3, 3}, {y, -3, 3}, {z, -3, 3}

]


11/97

u n i v e r s i t y

BAYLOR



5. Sketch x2 y2 z2= 1/2.


12/97

u n i v e r s i t y

BAYLOR



5. Sketch x2 y2 z2= 1/2.


13/97

u n i v e r s i t y

BAYLOR



5. Sketch x2 y2 z2= 1/2.

Over 600,000evaluations!


14/97

u n i v e r s i t y

BAYLOR



5. Sketch x2 y2 z2= 1/2.x2 y2 z2= 1/2


15/97

u n i v e r s i t y

BAYLOR


x2 y2 z2= 1/2


16/97

u n i v e r s i t y

BAYLOR


x2 y2 z2= 1/2

Polynomial equation


17/97

u n i v e r s i t y

BAYLOR


x2 y2 z2= 1/2

Polynomial equation


18/97

u n i v e r s i t y

BAYLOR


x2 y2 z2= 1/2

Polynomial equation


19/97

u n i v e r s i t y

BAYLOR


x2 y2 z2= 1/2

Polynomial equation

This is called a realalgebraic set or anaffine variety.

Th i i f f


20/97

u n i v e r s i t y

BAYLOR


The intersection of surfaces

Where do these surfaces intersect?

ATh i i f f


21/97

u n i v e r s i t y

BAYLOR




BAYLORTh i i f f


22/97

u n i v e r s i t y

BAYLOR




BAYLORTh i i f f


23/97

u n i v e r s i t y

BAYLOR




Intersection = Solutions =

BAYLORTh i t ti f f


24/97

u n i v e r s i t y

BAYLOR



Geometrically....

BAYLORTh i t ti f f


25/97

u n i v e r s i t y

BAYLOR



BAYLORTh i t ti f f


26/97

u n i v e r s i t y

BAYLOR



Secret : found with the cylindrical

algebraic decomposition (CAD) algorithm

BAYLORTh i t ti f f


27/97

u n i v e r s i t y

BAYLOR



BAYLORThe intersection of surfaces


28/97

u n i v e r s i t y

BAYLOR



BAYLORPolynomial ideals


29/97

u n i v e r s i t y

BAYLOR


Polynomial ideals

Notation : the collection of polynomials

with real coefficients is written



30/97

u n i v e r s i t y

BAYLOR


Polynomial ideals



Ring of polynomials



31/97

u n i v e r s i t y

BAYLOR


Polynomial ideals



Ring of polynomials

Defn : The idealgenerated by the npolynomials

is the set of polynomial combinations of thefks:



32/97

u n i v e r s i t y

BAYLOR


Polynomial ideals



Ring of polynomials





33/97

u n i v e r s i t y

BAYLOR


Polynomial ideals





34/97

u n i v e r s i t y

BAYLOR


Polynomial ideals



Giant set of polynomials.



35/97

u n i v e r s i t y

BAYLOR


Polynomial ideals




Think : vector space.



36/97

u n i v e r s i t y

BAYLOR


Polynomial ideals





Think : basis vectors.



37/97

u n i v e r s i t y

BAYLOR


Polynomial ideals






Fact : If are mpolynomials which generate

the same ideal, then



38/97

u n i v e r s i t y

BAYLOR


Polynomial ideals








Think : a different basis.



39/97

u n i v e r s i t y

BAYLOR


Polynomial ideals








Think : a different basis.



40/97

u n i v e r s i t y

BAYLOR


Polynomial ideals





41/97

u n i v e r s i t y

BAYLOR


Polynomial ideals





42/97

u n i v e r s i t y


Polynomial ideals



A Grbner basis is a good selection of thegks computed with Buchbergers algorithm

or Faugeres F4 algorithm



43/97

u n i v e r s i t y


Polynomial ideals





44/97

u n i v e r s i t y


Polynomial ideals



45/97

u n i v e r s i t y


Polynomial ideals

... help find exactglobal solutions byelimination(but even in moderately sized problems can become intractable)

Grbner bases can...

... identify if a polynomialfis in the ideal.

... allow for implicitization of a rationallyparameterized variety.

... compute related algebraic and geometric structures (e.g. the radical of the ideal)

u n i v e r s i t y

BAYLOR


46/97

u n i v e r s i t y


Differentialgeometrystudies smooth manifoldsvia

smooth topological structureAlgebraicgeometrystudies varietiesvia algebraic

structure

u n i v e r s i t y

BAYLOR


47/97

u n i v e r s i t y




structure

u n i v e r s i t y

BAYLOR


48/97

u n i v e r s i t y




structure

u n i v e r s i t y

BAYLOR


49/97

u n i v e r s i t y




structure

Real algebraic geometryis the study of real (semi)algebraic sets

u n i v e r s i t y

BAYLOR


50/97

u n i v e r s i t y



smooth topological structure

Algebraicgeometrystudies varietiesvia algebraic

structure

A semialgebraic setis a finite union of sets of the form

Real algebraic geometryis the study of real (semi)algebraic sets

u n i v e r s i t y

BAYLOR(Early?) Mantra of algebraic statistics


51/97

u n i v e r s i t y


Fact 1 : Statistical inference depends on thegeometryof the parameter space(the model)

( y ) g

E.g. regularity conditions, smoothness, etc.

u n i v e r s i t y



52/97



Fact 2 : Ifthe parameter spaceis a semialgebraic set,

then

statistical inference can be analyzed with

algebraic methods

( y ) g


u n i v e r s i t y



53/97



Fact 2 : Ifthe parameter spaceis a semialgebraic set,

then

statistical inference can be analyzed with

algebraic methods

( y ) g


Statistical models are algebraic varieties

u n i v e r s i t y

BAYLOREx 1 : A parametric ASM


54/97


A simple experiment

p

u n i v e r s i t y



55/97


A simple experiment

Requirements on the s

p

u n i v e r s i t y



56/97


A simple experiment

1.


p

u n i v e r s i t y



57/97


A simple experiment

1.

2.


u n i v e r s i t y



58/97


A simple experiment


1.

2.

u n i v e r s i t y



59/97



A simple experiment

1.

2.

This is a geometric structure!

u n i v e r s i t y



60/97



A simple experiment

1.

2.


u n i v e r s i t y

BAYLOR

Ex 1 : A parametric ASM


61/97



Statistical models = subsets of the simplex

u n i v e r s i t y

BAYLOR



62/97




Ex : The binomial distribution

u n i v e r s i t y

BAYLOR



63/97





u n i v e r s i t y

BAYLOR



64/97





u n i v e r s i t y

BAYLOR



65/97




Observation :

u n i v e r s i t y

BAYLOR



66/97




Observation :

Implic

itization

u n i v e r s i t y

BAYLOR



67/97




Observation :

Implic

itization

u n i v e r s i t y

BAYLOR



68/97




Observation :

Implicitizat

ion

u n i v e r s i t y

BAYLOR



69/97




Observation :

Implicitizat

ion

u n i v e r s i t y



70/97


The binomial model is the intersection of anaffine variety and the probability simplex!

u n i v e r s i t y



71/97


Two descriptions of the binomial model

Parametric description Implicit description

Were used to this... ...but sometimes we get this.

u n i v e r s i t y



72/97


Two descriptions of the binomial model


Were used to this... ...but sometimes we get this.


Possible (may be hard)

Not always possible

u n i v e r s i t y



73/97


A better visualization is available via

barycentric coordinates

3d to 2d

A last note : Visualization

... but since were insidethe simplex, wecant see the varieties outside of it

u n i v e r s i t y

BAYLOREx 2 : An implicitly defined ASM


74/97


The 2 x 2 contingency table

0 1

0

1

X2

X1

where

u n i v e r s i t y



75/97



0 1

0

1

X2

X1

where

u n i v e r s i t y



76/97



0 1

0

1

X2

X1

where


1.

2.

u n i v e r s i t y



77/97



0 1

0

1

X2

X1

where


1.

2. Points in/on the tetrahedronare distributions on the table!

u n i v e r s i t y



78/97



0 1

0

1

X2

X1

where


1.

2.

2d

3d

Points in/on the tetrahedronare distributions on the table!

u n i v e r s i t y



79/97


0 1

0

1

X2

X1

Ex : The independence model

3d



u n i v e r s i t y



80/97


0 1

0

1

X2

X1


3d


means , where is the marginal probability of X1 = x1:


u n i v e r s i t y



81/97


0 1

0

1

X2

X1


3d



This holds for everycombination of x1and x2!


u n i v e r s i t y



82/97


0 1

0

1

X2

X1


3d



This holds for everycombination of x1and x2!


u n i v e r s i t y



83/97


0 1

0

1

X2

X1


3d




u n i v e r s i t y



84/97


0 1

0

1

X2

X1

Ex : The independence modelStatistical models = subsets of the simplex



u n i v e r s i t y



85/97


0 1

0

1

X2

X1




u n i v e r s i t y



86/97


0 1

0

1

X2

X1




u n i v e r s i t y

BAYLOREx 3 : A Gaussian ASM


87/97


Do the inferential methods change?

For example, typically the MLE isasymptotically normal...

u n i v e r s i t y



88/97


Simulation :1. Draw n= 50, 1000 samples from

the bivariate normaldistribution with 0 mean.

2. Compute MLE for mean (closestpoint to the parameter space)

u n i v e r s i t y



89/97


One sample, its mean, and the MLE

u n i v e r s i t y



90/97


Simulation :1. Draw n= 50, 1000 samples from

the bivariate normaldistribution with 0 mean.

2. Compute MLE for mean (closestpoint to the parameter space)

Do N= 2000times

u n i v e r s i t yBAYLOREx 3 : A Gaussian ASM


91/97


n= 50



92/97


n= 1000



93/97


Do the inferential methods change?

For example, typically the MLE isasymptotically normal...

Conclusion :

Problems occur at singularities

which dont ever go away.

Problems happen near singularitieseven for relatively large n

u n i v e r s i t yBAYLOR


94/97


Algebraic statistical models

Defined in reference to another family of models (e.g. exponential family)

Defined as semialgebraic sets on the parameter space.



95/97


Algebraic statistical models

Defined in reference to another family of models (e.g. exponential family)

Defined as semialgebraic sets on the parameter space.

Drton and Sullivant, Statistica Sinica, 2007



96/97


Summary : Algebraic statistical models are an interestingand general class of models, but require care with many of theclassical aspects of inference...



97/97

Summary : Algebraic statistical models are an interestingand general class of models, but require care with many of theclassical aspects of inference...

... with cool pictures.

Thank you!

slides david kahle

Documents