mol evol jc.151216-amb.bayesianstats-teaser
Post on 08-Feb-2017
120 Views
Preview:
TRANSCRIPT
BayesianStatistics
MadeSimple
ÁlvaroMartínezBarrio,PhD
Alvaro.Martinez.Barrio@scilifelab.selinkedin.com/in/ambarrio@ambarrio
!Uppsala,Dec16th2015
2
Think Bayes
Bayesian Statistics Made Simple
Version 1.0.3
Allen B. Downey
Green Tea PressNeedham, Massachusetts
Notation:Probability
3
• p(A):theprobabilitythatAoccurs!
• p(A|B):theprobabilitythatAoccurs,giventhatBhasoccurred!
• p(AandB)=p(A)p(B|A):Conjointprobability
Introduction:Bayes’Theorem
4
• Bydefinitionofconjointprobabilityandthatconjunctioniscommutative:p(AandB)=p(A)p(B|A)=(1)p(BandA)=p(B)p(A|B)
• p(A)p(B|A)=p(B)p(A|B)(2)• p(A|B)=p(A)p(B|A)/p(B)(3)
Thecookieproblem
5
Supposetherearetwobowlsofcookies.Bowl1contains30vanillacookiesand10chocolatecookies.Bowl2contains20ofeach.Nowsupposeyouchooseoneofthebowlsatrandomand,withoutlooking,selectacookieatrandom.Thecookieisvanilla.Whatisthe
probabilitythatitcamefromBowl1?
Thecookieproblem
6
Supposetherearetwobowlsofcookies.Bowl1contains30vanillacookiesand10chocolatecookies.Bowl2contains20ofeach.Nowsupposeyouchooseoneofthebowlsatrandomand,withoutlooking,selectacookieatrandom.Thecookieisvanilla.Whatisthe
probabilitythatitcamefromBowl1?
p(B1|V)=p(B1)p(V|B1)/p(V)
Thecookieproblem
7
Supposetherearetwobowlsofcookies.Bowl1contains30vanillacookiesand10chocolatecookies.Bowl2contains20ofeach.Nowsupposeyouchooseoneofthebowlsatrandomand,withoutlooking,selectacookieatrandom.Thecookieisvanilla.Whatisthe
probabilitythatitcamefromBowl1?
p(B1|V)=p(B1)p(V|B1)/p(V)
p(B1|V)=(1/2)(3/4)/5/8
History:Bayes’Theorem
8
ThomasBayes,
(b.1702,London-d.1761,
TunbridgeWells,Kent)
Intheearly18thcentury,themathematiciansofthetimeknewhowtofindtheprobabilitythat,say,4peopleaged50dieinagivenyearoutofasampleof60iftheprobabilityofanyoneofthemdyingwasknown.
Buttheydidnotknowhowtofindtheprobabilityofone50-yearolddyingbasedontheobservationthat4haddiedoutof60.
History:Bayes’Theorem
9
ThomasBayes,
(b.1702,London-d.1761,
TunbridgeWells,Kent)
Intheearly18thcentury,themathematiciansofthetimeknewhowtofindtheprobabilitythat,say,4peopleaged50dieinagivenyearoutofasampleof60iftheprobabilityofanyoneofthemdyingwasknown.
Buttheydidnotknowhowtofindtheprobabilityofone50-yearolddyingbasedontheobservationthat4haddiedoutof60.
thequestionofinverseprobability
The“diachronic”interpretation
10
ThomasBayes,
(b.1702,London-d.1761,
TunbridgeWells,Kent)
p(H|D)=p(H)p(D|H)/p(D)
• p(H)istheprobabilityofthehypothesisbeforeweseethedata,calledthepriorprobability,orjustprior.
• p(H|D)iswhatwewanttocompute,theprobabilityofthehypothesisa\erweseethedata,calledtheposterior.
• p(D|H)istheprobabilityofthedataunderthehypothesis,calledthelikelihood.
• p(D)istheprobabilityofthedataunderanyhypothesis,calledthenormalizingconstant.
History:Syllogism
11
4thcenturyBC
!!
• Majorpremise!!
• Minorpremise!!
• Conclusion
Arhetoricalsyllogism(a3-partdeductiveargument)usedinoratorialpractice.
History:Syllogism
12
4thcenturyBC
• Majorpremise:“Allhumansaremortal”!!
• Minorpremise:“AllGreeksarehuman”!
• Conclusion:“AllGreeksaremortal”
History:Syllogism
13
4thcenturyBC
• Majorpremise:“Allmortalsdie”!!
• Minorpremise:“Allmenaremortals”!
• Conclusion:“Allmendie”
History:Enthymeme
14
4thcenturyBC
!• “Socratesismortalbecausehe’s
human”
!• Majorpremise(unstated):“Allhumansaremortal.”!
• Minorpremise(stated):“Socratesishuman.”!
• Conclusion(stated):“Therefore,Socratesismortal.”
History:Enthymeme
15
4thcenturyBC
!• "Heisill,sincehehasacough.”
!!
• “Sinceshehasachild,shehas
givenbirth."
History:Enthymeme
16
4thcenturyBC
• Hestartedtoproposethatenthymemesarebasedonprobabilities(eikos),examples,tekmêria(i.e.,proofs,evidences),andsigns(sêmeia).
History:Enthymeme
17
4thcenturyBC
• CarolPosterarguesthatenthymemesastruncatedsyllogismswasinventedbyBritishrhetoricians(suchasRichardWhately)intheXVIIIcentury.
Poster,Carol(2003)."Theology,Canonicity,andAbbreviatedEnthymemes".RhetoricSocietyQuarterly33(1):67–103.
Chapter1:Bayes’Theorem
• Mutuallyexclusive:Atmostonehypothesisinthesetcanbetrue!
• Collectivelyexhaustive:Therearenootherpossibilities;atleastoneofthehypotheseshastobetrue
Chapter1:Bayes’Theorem
• Mutuallyexclusive:Atmostonehypothesisinthesetcanbetrue!
• Collectivelyexhaustive:Therearenootherpossibilities;atleastoneofthehypotheseshastobetrue
p(D)=p(B1)p(D|B1)+p(B2)p(D|B2)
Chapter1:Bayes’Theorem
• Mutuallyexclusive:Atmostonehypothesisinthesetcanbetrue!
• Collectivelyexhaustive:Therearenootherpossibilities;atleastoneofthehypotheseshastobetrue
p(D)=p(B1)p(D|B1)+p(B2)p(D|B2)
p(D)=(1/2)(3/4)+(1/2)(1/2)=5/8
Chapter1:Bayes’Theorem
• Mutuallyexclusive:Atmostonehypothesisinthesetcanbetrue!
• Collectivelyexhaustive:Therearenootherpossibilities;atleastoneofthehypotheseshastobetrue
p(D)=p(B1)p(D|B1)+p(B2)p(D|B2)
p(D)=(1/2)(3/4)+(1/2)(1/2)=5/8
Ifp(A|B)ishardtocompute,orhardtomeasureexperimentally,checkwhetherit
mightbeeasiertocomputetheothertermsinBayes’stheorem,p(B|A),p(A)andp(B).
Chapter2:ComputationalStatistics
• Distribution:
Chapter2:ComputationalStatistics
• Distribution:setofvaluesandtheircorrespondingprobabilities.
Chapter2:ComputationalStatistics
• Distribution:setofvaluesandtheircorrespondingprobabilities.• Probabilitymassfunction:waytorepresentadistributionmathematically.
Chapter2:ComputationalStatistics
• Distribution:setofvaluesandtheircorrespondingprobabilities.• Probabilitymassfunction:waytorepresentadistributionmathematically.
• Whentalkingaboutprobabilities,youneedtonormalise(theyshouldaddupto1)
Chapter2:ComputationalStatistics
• Distribution:setofvaluesandtheircorrespondingprobabilities.• Probabilitymassfunction:waytorepresentadistributionmathematically.
• Whentalkingaboutprobabilities,youneedtonormalise(theyshouldaddupto1)
• Thisdistribution,whichcontainsthepriorsforeachhypothesis,iscalled(waitforit)thepriordistribution.
• Toupdatethedistributionbasedonnewdata(avanillacookie!),wemultiplyeachpriorbythecorrespondinglikelihood.
• Thedistributionisnolongernormalized,youneedtorenormalize• Theresultisadistributionthatcontainstheposteriorprobabilityforeachhypothesis,whichiscalled(waitagain!)theposteriordistribution.
Chapter2:ComputationalStatistics
• Distribution:setofvaluesandtheircorrespondingprobabilities.• Probabilitymassfunction:waytorepresentadistributionmathematically.
• Whentalkingaboutprobabilities,youneedtonormalise(theyshouldaddupto1)
• Thisdistribution,whichcontainsthepriorsforeachhypothesis,iscalled(waitforit)thepriordistribution.
• Toupdatethedistributionbasedonnewdata(avanillacookie!),wemultiplyeachpriorbythecorrespondinglikelihood.
• Thedistributionisnolongernormalized,youneedtorenormalize• Theresultisadistributionthatcontainstheposteriorprobabilityforeachhypothesis,whichiscalled(waitagain!)theposteriordistribution.
Terminologyanddesignpatternsofpythonprogramsthatyoucanuseduringtherest
ofthecourse
Chapter3:Estimation
SupposeIhaveaboxofdicethatcontainsa4-sideddie,a6-sideddie,an8-sideddie,a12-sideddie,anda20-sideddie.IfyouhaveeverplayedDungeons&Dragons,youknowwhatIamtalkingabout.SupposeIselectadiefromtheboxatrandom,rollit,andgeta6.WhatistheprobabilitythatIrolledeachdie?
Chapter3:Estimation
SupposeIhaveaboxofdicethatcontainsa4-sideddie,a6-sideddie,an8-sideddie,a12-sideddie,anda20-sideddie.IfyouhaveeverplayedDungeons&Dragons,youknowwhatIamtalkingabout.SupposeIselectadiefromtheboxatrandom,rollit,andgeta6.WhatistheprobabilitythatIrolledeachdie?
Letmesuggestathree-stepstrategyforapproachingaproblemlikethis:1.Choosearepresentationforthehypotheses. 2.Choosearepresentationforthedata. 3.Writethelikelihoodfunction.
Chapter3:Estimation
Mosteller’sFiftyChallengingProblemsinProbabilitywithSolutions
“Arailroadnumbersitslocomotivesinorder1..N.Onedayyouseealocomotivewiththenumber60.Estimatehowmanylocomotives
therailroadhas.”
Chapter3:Estimation
PartIofStatisticalInference.
“Arailroadnumbersitslocomotivesinorder1..N.Onedayyouseealocomotivewiththenumber60.Estimatehowmanylocomotives
therailroadhas.”
Chapter3:Estimation
PartIofStatisticalInference.
“Arailroadnumbersitslocomotivesinorder1..N.Onedayyouseealocomotivewiththenumber60.Estimatehowmanylocomotives
therailroadhas.”
Chapter3:Estimation
PartIofStatisticalInference.
“Arailroadnumbersitslocomotivesinorder1..N.Onedayyouseealocomotivewiththenumber60.Estimatehowmanylocomotives
therailroadhas.”
Chapter3:Estimation
PartIofStatisticalInference.
“Arailroadnumbersitslocomotivesinorder1..N.Onedayyouseealocomotivewiththenumber60.Estimatehowmanylocomotives
therailroadhas.”
Therearetwowaystoproceed:!
•Getmoredata. •Getmorebackgroundinformation.
Chapter3:Estimation
PartIofStatisticalInference.
“Arailroadnumbersitslocomotivesinorder1..N.Onedayyouseealocomotivewiththenumber60.Estimatehowmanylocomotives
therailroadhas.”
Therearetwowaystoproceed:!
•Getmoredata. •Getmorebackgroundinformation.
Chapter3:Estimation
PartIofStatisticalInference.
“Arailroadnumbersitslocomotivesinorder1..N.Onedayyouseealocomotivewiththenumber60.Estimatehowmanylocomotives
therailroadhas.”
Therearetwowaystoproceed:!
•Getmoredata. •Getmorebackgroundinformation.
Chapter3:Estimation
PartIofStatisticalInference.
“Arailroadnumbersitslocomotivesinorder1..N.Onedayyouseealocomotivewiththenumber60.Estimatehowmanylocomotives
therailroadhas.”
Therearetwowaystoproceed:!
•Getmoredata. •Getmorebackgroundinformation.
Chapter3:Estimation
PartIofStatisticalInference.
“Arailroadnumbersitslocomotivesinorder1..N.Onedayyouseealocomotivewiththenumber60.Estimatehowmanylocomotives
therailroadhas.”
Therearetwowaystoproceed:!
•Getmoredata. •Getmorebackgroundinformation.
Chapter3:Estimation
• Credibleinterval:Forintervalsweusuallyreporttwovaluescomputedsothatthereisa90%chancethattheunknownvaluefallsbetweenthem(oranyotherprobability).
• Thewidthofthisintervalsuggestshowuncertainweareabouttheconclusionbasedinourunknownvalue.
• Therearetwoapproachestochoosingpriordistributions:• i)informative:bestrepresentsbackgroundinformation• ii)uninformative:intendedtobeasunrestrictedaspossible
Chapter3:Estimation
• Credibleinterval:Forintervalsweusuallyreporttwovaluescomputedsothatthereisa90%chancethattheunknownvaluefallsbetweenthem(oranyotherprobability).
• Thewidthofthisintervalsuggestshowuncertainweareabouttheconclusionbasedinourunknownvalue.
• Therearetwoapproachestochoosingpriordistributions:• i)informative:bestrepresentsbackgroundinformation• ii)uninformative:intendedtobeasunrestrictedaspossible
Inrealworldyouhavetwowaystoproceed:!
Ifyouhavealotofdata,thechoiceofthepriordoesn’tmatterverymuch;informativeanduninformativepriorsyieldalmostthesameresults.
!Ifyoudon’thavemuchdata,usingrelevantbackgroundinformationmakesabig
difference.
DifferencesbetweenBayesiansandNon-Bayesians
AccordingtoJeffGill(CenterforAppliedStatistics,WashU)
DifferencesbetweenBayesiansandNon-Bayesians
AccordingtoJeffGill(CenterforAppliedStatistics,WashU)
ACCP 37th Annual Meeting, Philadelphia, PA [2]
Differences Between Bayesians and Non-BayesiansAccording to my friend Jeff Gill
Typical Bayesian Typical Non-BayesianTypicalBayesian
DifferencesbetweenBayesiansandNon-Bayesians
ACCP 37th Annual Meeting, Philadelphia, PA [2]
Differences Between Bayesians and Non-BayesiansAccording to my friend Jeff Gill
Typical Bayesian Typical Non-BayesianTypicalBayesian
ACCP 37th Annual Meeting, Philadelphia, PA [2]
Differences Between Bayesians and Non-BayesiansAccording to my friend Jeff Gill
Typical Bayesian Typical Non-BayesianTypicalNon-Bayesian
AccordingtoJeffGill(CenterforAppliedStatistics,WashU)
Conclusions
• Importanceofmodelling• Followadiscreteapproach:correctfirst,andexpandlater
GeneralApproach
1.Startwithsimplemodelsandimplementtheminclear,readableanddemonstrablycorrectcode.Focusshouldbeongoodmodellingdecisions,notoptimisation
GeneralApproach
1.Startwithsimplemodelsandimplementtheminclear,readableanddemonstrablycorrectcode.Focusshouldbeongoodmodellingdecisions,notoptimisation2.Identifythebiggestsourcesoferror.Perhapsincreasethenumberofvaluesinadiscreteapproximation,increasethenumberofiterationsinaMCsimulation,oradddetailstothemodel
GeneralApproach
1.Startwithsimplemodelsandimplementtheminclear,readableanddemonstrablycorrectcode.Focusshouldbeongoodmodellingdecisions,notoptimisation2.Identifythebiggestsourcesoferror.Perhapsincreasethenumberofvaluesinadiscreteapproximation,increasethenumberofiterationsinaMCsimulation,oradddetailstothemodel3.Isperformancegood?Ifnot,tryoptimisingthen
REFERENCES
• PyContutorials(byAllenDowney)https://sites.google.com/site/simplebayes/!• “ProbablyOverthinkingIt”(byAllenDowney)http://allendowney.blogspot.se/!
• MarkA.Beaumont&BruceRannala(2004)NatureRevGenetics
MonumenttomembersoftheBayesandCottonfamilies,includingThomasBayesandhisfatherJoshua,inBunhillFieldsburialground
top related