bayesian network : an introduction may 2005 김 진형 kaist...
TRANSCRIPT
2
BN = graph theory + probability theory
Qualitative part: graph theoryDirected acyclic graph Nodes: variables Edges: dependency or influence of a variable on another.
Quantitative part: probability theory
Set of conditional probabilities for all variables
Naturally handles the problem of complexity and uncertainty.
3
Bayesian Network is
A framework for representing uncertainty in our knowledgeA Graphical modeling framework of causality and influenceA Representation of the dependencies among random variablesA compact representation of a joint probability of variables on the basis of the concept of conditional independence.
Earthquake
Radio Alarm
Burglar
Alarm-Example
4
Bayesian Network Syntax
A set of nodes, one per variableA diected, acyclic graph (link = “directly influences”)A conditional distribution for each node given its parents : P(Xi| Parents(Xi))
In the simplist case, conditional distribution represented as a conditional probability table (CPT) giving the distribution over Xi for each combination of parent values
5
Earthquake Example
I’m at work, neighbor John calls to say my alarmis ringing, but neighbor Mary doesn’t call. Sometimes it’s set off by minor earthquakes. Is there a burglar ?Variable : Burglar, Earthquake, Alarm, JohnCalls, MaryCallsNetwork Topology
A burglar can set the alarm offAn earthquake can set the alarm offThe alarm can cause Mary to callThe alarm can cause John to call
6
Earthquake Example
7
Representation of Joint Probability
Joint probability as a product of conditional probabilities
Can dramatically reduce the parameters for data modeling in Bayesian networks.
C
E
D
BA
))(|())(|(
))(|())(|())(|(),,,,(
EEPDDP
CCPBBPAAPEDCBAP
PaPa
PaPaPa
8
Causal Networks
Node: event
Arc: causal relationship between two nodesA B: A causes B.
Causal network for the car start problem [Jensen 01]
Fuel
Fuel MeterStanding Start
Clean SparkPlugs
9
Reasoning with Causal Networks
• My car does not start. increases the certainty of no fuel and dirty spark plugs. increases the certainty of fuel meter’s standing for the empty.
• Fuel meter stands for the half. decreases the certainty of no fuel increases the certainty of dirty spark plugs.
Fuel
Fuel MeterStanding
Start
Clean SparkPlugs
10
Structuring Bayesian Network
Initial configuration of Bayesian NetworkRoot nodes
Prior probabilities
Non-root nodesConditional probabilities given all possible combinations of direct predecessors
A B
D
E
C
P(b)P(a)
P(d|ab), P(d|a ㄱ b), P(d| ㄱ ab), P(d| ㄱ a ㄱ b)
P(e|d)
P(e| ㄱ d)
P(c|a)
P(c| ㄱa)
11
Structuring Bayesian Network
Fuel
Fuel MeterStanding
Start
Clean SparkPlugs
P(Fu = Yes) = 0.98 P(CSP = Yes) = 0.96
P(St|Fu, CSP)P(FMS|Fu)
0.001
0.60
FMS = Half
0.9980.001Fu = No
0.010.39Fu = Yes
FMS = Empty
FMS = Full
10(No, Yes)
0.990.01(Yes, No)
10(No, No)
0.010.99(Yes, Yes)
Start=NoStart=YES(Fu, CSP)
12
Independence assumptions & Complexity
Problem of probability theory2n-1 joint distributions for n variables
For 5 variables, 31 joint distributions
Solution by BNFor 5 variables, 10 joint distributions
Bayesian Networks have built-in independence
assumptions.
A B
D
E
C
P(b)P(a)
P(d|ab), P(d|a ㄱ b), P(d| ㄱ ab), P(d| ㄱ a ㄱ b)
P(e|d)
P(e| ㄱ d)
P(c|a)
P(c| ㄱa)
13
Independent Assumptions
CA BC
A B
A and B is marginally dependent
CA BC
A B
A and B is conditionally independent
C
A B
A and B is marginally independent
C
A B
A and B is conditionally dependent
14
Independent Assumption : Car Start Problem
1. ‘Start’ and ‘Fuel’ are dependent on each other.
2. ‘Start’ and ‘Clean Spark Plugs’ are dependent on each other.
3. ‘Fuel’ and ‘Fuel Meter Standing’ are dependent on each other.
4. ‘Fuel’ and ‘Clean Spark Plugs’ are conditionally dependent on each other given the value of ‘Start’.
5. ‘Fuel Meter Standing’ and ‘Start’ are conditionally independent given the value of ‘Fuel’.
Fuel
Fuel MeterStanding
Start
Clean SparkPlugs
15
Quantitative Specification by Probability Calculus
FundamentalsConditional Probability
Product Rule
Chain Rule: a successive application of the product rule.
)(
),()|(
AP
BAPABP
)()|()()|(),( BPBAPAPABPBAP
n
iii
nnnnn
nnn
n
XXXPXP
XXXXPXXXXPXXXP
XXXXPXXXP
XXXP
2111
1212211221
121121
21
),...,|()(
),...,,|(),...,,|(),...,,(
),...,,|(),...,,(
),...,,(
16
Main Issues in BN
Inference in Bayesian networksGiven an assignment of a subset of variables (evidence) in a BN, estimate the posterior distribution over another subset of unobserved variables of interest.
Learning Bayesian network from dataParameter Learning
Given a data set, estimate local probability distributions P(Xi|Pa(Xi)). for all variables (nodes) comprising the BN .
Structure learningFor a data set, search a network structure G (dependency structure) which is best or at least plausible.
obs
obsunobsun P
PP
x
xxxx
,)|(
17
Evaluating networks
Evaluation of network (inference)Computation of all node’s conditional probability given evidence
Type of evaluationExact inference
NP-Hard Problem
Approximate inferenceNot exact, but within small distance of the correct answer
18
Inference Task in Bayesian networks
19
Inference in Bayesian networks
Joint distribution
Definition of joint distributionSet of boolean variables (a,b)
P(ab), P( ㄱ ab), P(a ㄱ b), P( ㄱ a ㄱ b)
Role of joint distributionJoint distribution give all the information about probability distribution.
Ex> P(a|b) = P(ab) / P(b)
= P(ab) / ((P(ab)+P( ㄱ ab))
For n random variables, 2n –1 joint distributions
20
Inference in Bayesian networks
Joint distribution for BN is uniquely definedBy the product individual distribution of R.V.
Using chain-rule, topological sort and dependency
ba
c d
e
P(abcde) = P(a)P(b)P(c|a)P(d|ab)P(e|d)
Ex)
21
Inference in Bayesian networks
Example
ba
c d
e
P(a)P(b|a)P(c|ab)P(d|abc)P(e|abcd)
Joint probability P(abcde)
Chain-rule,
Topological sort
P(abcde) = P(a)P(b)P(c|a)P(d|ab)P(e|d)
Independence assumption
b is independent on a,c
d is independent on c
e is independent on a,b,c
22
Exact inference
Two network typesSingly connected network (polytree)
Multiply connected network
Complexity according to network typeSingly connected network can be efficiently solved
E
C D
A B
Singly connected network
A
B C
D
Multiply connected network
23
Inference by enumeration
24
Evaluation Tree
25
Exact inference
Multiply Connected Network
Hard to evaluate multiply connection network
A
B C
DD will affect C directly
D will affect C indirectly
p(C|D) ?
evidence
Probabilities can be affected by both neighbor nodes and other nodes
26
Exact inference
Multiply Connected Network (cont.)
Methodology to evaluate the network exactly
ClusteringTo Combination of nodes until the resulting graph is singly connected
Cloudy
WetGrass
Spr+Rain
C P(S=F) P(S=T)
F
T
0.5 0.5
0.9 0.1
C P(R=F) P(R=T)
F
T
0.8 0.2
0.2 0.8
C P(S,R)
FF FT TF TT
F
T
.40 .10 .40 .10
.18 .72 .02 .08
C
W
RS
27
Inference by Stochastic Simulation
28
Sampling from an empty network
29
Real World Applications of BN
Intelligent agentsMicrosoft Office assistant: Bayesian user modeling
Medical diagnosisPATHFINDER (Heckerman, 1992): diagnosis of lymph node disease commercialized as INTELLIPATH (http://www.intellipath.com/)
Control decision support systemSpeech recognition (HMMs)Genome data analysis
gene expression, DNA sequence, a combined analysis of heterogeneous data.
Turbocodes (channel coding)
30
MSBNx
Tools for Baysian Network building tool programmed by Microsoft research
Free downloadablehttp://research.microsoft.com/adapt/MSBNx/
FeaturesGraphical Editing of Bayesian NetworksExact Probability Calculations XML Format MSBN3 ActiveX DLL provides an COM-based API for editing and evaluating Bayesian Networks.
31
Conclusion
Bayesian Networks are solutions of problems in traditional probability theory
Reason of using BNBN need not many numbers
Efficient exact solution methods as well as a variety of approximation schemes