influence diagram

Influence Diagram

主講人：虞台文

大同大學資工所智慧型多媒體研究室

Content

IntroductionDefinition & Optimal PolicyFinding Optimal Policy

Influence Diagram

Introduction

大同大學資工所智慧型多媒體研究室

Decision Analysis

Decision Tree --- Traditional approaches– Tree size grow exponentially with the number of deci

sions and attributes

Influence Diagram– Introduced by Harward and Matheson on 1984.– An extension of Bayesian Networks– More compact than decision tree– Reveal more problem structure in decision making

Influence Diagrams

An influence Diagram is an acyclic graph with three types of nodes– Random nodes

• A Bayesian Network only contains random nodes

– Decision nodes– Value nodes

Example

Sick’ Dry’

Loses’

Sick Dry

Loses

Treat

Harv

Cost

Random node

Decision node

Value node

Example

Sick’ Dry’

Loses’

Sick Dry

Loses

Treat

Harv

Cost

Sick Probability

sick 0.1not 0.9

Dry Probability

dry 0.1not 0.9

Losesdry not

sick not sick not

yes 0.95 0.85 0.9 0.02

not 0.05 0.15 0.1 0.98

Example

Sick’ Dry’

Loses’

Sick Dry

Loses

Treat

Harv

Cost

Sick’treat not

sick not sick not

sick 0.2 0.01 0.99 0.02

not 0.8 0.99 0.01 0.98

Treattreatnot

Dry’ dry not

dry 0.6 0.05

not 0.4 0.95Loses’

dry not

sick not sick not

yes 0.95 0.9 0.85 0.02

not 0.05 0.1 0.15 0.98

Example

Sick’ Dry’

Loses’

Sick Dry

Loses

Treat

Harv

Cost

Treat treat not

Cost/Utility -8000 0

Harv sick not

Harv/Utility 3000 20000

Example

Sick’ Dry’

Loses’

Sick Dry

Loses

Treat

Harv

Cost

Observe that the apple tree loses leaves.

Whether the tree will be treated?

DemoDemo

Influence Diagram

Definition & Optimal Policy

大同大學資工所智慧型多媒體研究室

Nodes of Influence Diagrams

Random (Chance) nodes– Each represent a random variable whose value is dictated by

some probability distribution.– Each is associated with a conditional probability distribution.

Decision nodes– Each represents decision variable whose value is to be

chosen by the decision maker.– Each is associated with a policy.

Value nodes– Each represents a real-valued utility function– Can’t have child.

( )vvf

Arcs of Influence Diagrams

Conditional Arcs– Arcs into random nodes.

Informational Arcs– Arcs into decision nodes.

Notations

x

v

X DC U The set of all value nodes

The set of all decision nodes

The set of all chance nodes

: Parents of x X.

: The frame of v CD.

, J xx JJ C D

Random Nodes

ci

. . .

c C D Conditional

Arcs

( | ), i ii c i cP c c

Decision Nodes

. . .id C D

InformationalArcs

di

Decision function

:d ii

i d Depends on policy

What is the optimal policy?

How many policies we may choose?

That is, what is the size of decision space?

Decision Nodes

. . .id C D

InformationalArcs

di

Decision function

:d ii

i d

A BN with decision nodes d1,…,dk.

Policy 1( , , )k

Given a policy, each decision nodes can be converted to a random node by

1 if ( )( | )

0 otherwisei

i i

i d ii d

dP d

Decision Nodes

. . .id C D

InformationalArcs

di

Decision function

:d ii

i d

A BN with decision nodes d1,…,dk.

Policy 1( , , )k

Given a policy, each decision nodes can be converted to a random node by

1 if ( )( | )

0 otherwisei

i i

i d ii d

dP d

Thus, given a policy , an inference diagram with random nodes and decision nodes only can be viewed as a BN.

Thus, given a policy , an inference diagram with random nodes and decision nodes only can be viewed as a BN.

Probability Evaluation

Given a policy, say, =(1, …, k), the probability with nodes CD is then

1 if ( )( | )

0 otherwisei

i i

i d ii d

dP d

( , ) ( | ) ( | )d Dc

c dC

P P PD dC c

A CD then

( ) ( , )D AC

CP A DP

Utility Nodes

. . .iv C D

Utility function

:i vi

vf R vi

iv U

Expected Values

Given a policy =(1, …, k), the expected value of utility node v in the influence diagram is

:i vi

vf R

( )[ (] )v

v v vfE Pv

The expected value of the influence diagram N is

[ ] ( )v U

E E v

N

Optimal Expected Value

[ ] ( )v U

E E v

N

OptimalExpected Value

[ ] max [ ]E E N N

Optimal Policy * arg max [ ]E N

*[ ] [ ]E E N NHow?

Influence Diagram

Finding Optimal Policy

大同大學資工所智慧型多媒體研究室

Shachter and Peot’s Transformation

Convert a value node v to a binary random node

( )[ (] )v

v v vfE Pv

( )( 1| ) v v

vv

fP v

M

max ( )vv v vM f

Given a policy, denote the reformulated BN as N.

Shachter and Peot’s Transformation

( )[ (] )v

v v vfE Pv

( )( 1| ) v v

vv

fP v

M

( )

( 1| ) v vv

v

fP v

M

{0,1}

[ ] ( )v

E v vP v

N N 0 ( 0) 1 ( 1)P v P v

N N

( 1)P v

N

. . .

v

( 1| ) ( )v

v vP v P

1

( ) ( )v

v v vv

f PM

1

( )v

E vM ( ) ( 1) vE v P v M

N( ) ( 1) vE v P v M

N

Shachter and Peot’s Transformation

[ ] [ ]v U

E E v

N

( ) ( 1) vE v P v M N

( ) ( 1) vE v P v M N

( 1) vv U

MP v

N

[ ] max [ ]E E v N (max 1)v U

vP v M

N

Evaluating the expected value for a given policy:

Evaluating the expected value for an optimal policy:

Bucket Elimination Algorithm