Contingency Assessment Methods & Trends

Dr Stephen Grey, Associate Director, BroadleafMarch 19, 2014

Contents

Overview

Monte Carlo simulation

Context

How we got here

Two big problems

Alternative approach

2

Overview

Quantitative evaluation of project cost contingency has become stuck in outdated methods

Common practice is not good practice

Project size and complexity took off too fast for computing power and availability to keep up

Manual methods became automated rather than better methods being taken up when computing became more accessible

We can now adopt better practice and make quantitative analysis more efficient and realistic

Monte Carlo simulationVery brief summary

Distributions represent values and probabilities represent events

Thousands of examples of possible outcomes sampling distributions and calculating outcomes

Interpret the results as an indication of what could happen in reality

Cost

No effect

33%

67%

$4MEvent

Model

Impossible

Risky Manageable

Safe

Context

5

Estimate $X $C+

QuantitativeRiskAssessment

No standards

MethodMethodMethodMethod? ?

Entrainedthinking

How did we get here?

6

1960

1970

1980

1990

2000

2010

Projects used to organise workCritical Path Method

PROJECTS COMPUTING

Bureau facilities - specialist systems designers and programmers

Work Breakdown StructureCPM on large jobs

Some users coding overnight turnaroundLaborious computer CPM input

Projects become more complicatedespecially in IT and communications

Personal computing for someIndividual access to CPM and MCS

Projects become bigger Personal computing everywhereMCS widespread

Methods of assessing contingency

Model structures Risk events Line items Risk factors

Calculation Manual Computer assisted Monte Carlo simulation with distributions

7

Risk event models

No effect

Probability = 33%

Probability = 67%

$4M$2M $8MONE EVENT

Expected value = 33% x $4MExpected value = 33% x Mean

Risk event models

ALL THE RISKS

RISK Probability Impact PxIRisk1 P1 I1 P1 xI1Risk2 P2 I2 P2 xI2Risk3 P3 I3 P3 xI3Risk4 P4 I4 P4 xI4

Riskn Pn In Pn xInTotal Pi xIiM

onte

Car

lo s

imul

atio

n

Line item models

WBS ITEM Labour Materials Total ContingencyItem 1 $ $ $ $Item 2 $ $ $ $Item 3 $ $ $ $Item 4 $ $ $ $ Item N $ $ $ $Total $ $ M

onte

Car

lo s

imul

atio

n

ProjectestimatesummaryLabour Facilities Super

visionMaterials Sub

contractsServices Expenses Total

Earthworks 1234567 1234567 1234567 1234567 1234567 1234567 1234567 ?

Concrete 1234567 1234567 1234567 1234567 1234567 1234567 1234567 ?

1234567 1234567 1234567 1234567 1234567 1234567 1234567 ?

Overheads 1234567 1234567 1234567 1234567 1234567 1234567 1234567 ?

ProjectTotal ? ? ? ? ? ? ? ?

Risk factors model

Uncertainty about quantities of concrete

(m3)

Uncertainty about rates for cost of concrete ($/m3)

x x

The only practical way to evaluate a risk factor modelis to use Monte Carlo simulation on a computer

Cost estimating relationships (e.g. Cost = Quantity x Unit rate)

How timing affected what we do now

1960

1970

1980

1990

2000

2010

RISK EVENTS LINE ITEMS RISK FACTORS

All we have knownMust be rightDe facto standard

World BankPouliquen et al

Six monthsto build

simulation

Why the risk factor approach?Problems with risk event models

Many risks are not really events but variations from the estimating assumptions vary continuously across a range Staff productivity Rates for office space How long the work will go on and incur overheads

Some risks affect more than one part of the cost

One part of the cost will be affected by more than one risk

Two or more risks will often interact

Trying to represent uncertainties as discrete separate events is inefficient and confusing

Risk factors model process plant construction

COSTS

RISK FACTORS

RISKS

Concrete quantity

Concrete $Steel $

Steel quantity

One riskaffecting

two factors Two risksaffecting

one factor

Vibrationcharacteristics

Geotechnicalcharacteristics

Other factor(s) Other factor(s)

Other risks

Risk factors model IT system roll out

COSTS

RISK FACTORS

RISKS

Unit cost of licenses

License costs $

Professional services $

Roll out effort

One riskaffecting

two factors Two risksaffecting

one factor

Decision onoperating system

Commercialarrangementsfor licenses

Other factor(s) Other factor(s)

Other risks

Why the risk factor approach?Problems with line item models

Separate lines are often not independent of one another Steel cost uncertainty will affect every item with steel in it Salary cost uncertainty will affect every item with staff costs

In reality line item uncertainties are correlated

In principle, it is technically feasible to model the correlations realistically in a line item structure but its not practicable

Line item models often either understate the uncertainty in the total cost, because they ignore correlation, or lack credibility due to relying on correlations that cannot be justified

Effect of missing correlation

Realisticcorrelation

Correlationmissing

Target cost

Like

lihoo

d of

aris

ing

False sense of accuracy and confidence

Why does missing correlation matter?

Target cost

Ris

k of

exc

eedi

ng ta

rget

Required level of confidence

Understatement of funding requirements

Risk factorsKey points

Based on cost estimating relationships that people already understand (other effects and relationships where required)

Draw together the effects of risks that overlap

Represent explicitly the relationships that cause correlations

Represent the interactions between uncertainties

Capture the continuous nature of variations

Relatively simple models that are realistic and easy to understand (typically 20-40 factors)

Evaluated using Monte Carlo simulation

Early example of risk factors modelPouliquen, L.Y., 1970, Risk Analysis in Project Appraisal, World Bank Staff Occasional Papers, No. 11, Johns Hopkins Press, Baltimore

Ris

k fa

ctor

s

Key

fact

orsTotal

Distribution parameters

Assessing the range on one risk factorAvoid bias, document analysis

Assumptions embodied in base estimate (key ones)

Sources of uncertainty

Pessimistic and optimistic scenarios

Range of possible outcomes (P10, Most Likely, P90)

Example of a risk factor assessmentRate for office space for project team ($/m2/mth)

Assumptions Rate based on informal enquiries with property agents X square metres per person plus 10% allowance Lease starting mid next year and running for 24mth

Sources of uncertainty Market demand in the 2-3mth before lease starts Ability to obtain suitable space matched to team size, which

is still uncertain, affecting space utilisation (wasted space)

Example of a risk factor assessmentRate for office space for project team ($/m2/mth)

Pessimistic scenario Market heats up Have to lease bigger space than bare minimum required, up

to 25% more than strictly required for team size

Optimistic scenario Market cools Find space that closely matches team size

Example of a risk factor assessmentRate for office space for project team ($/m2/mth)

Pessimistic estimate (P90) Market rises ~15% Wasted space adds ~25% Net effect ~1.15 x ~1.25 = ~1.44, say 50% above estimate

Optimistic estimate (P10) Market eases ~10% Save about half the 10% allowance for wasted space Net effect ~0.9 x ~0.95 = ~0.86, say 15% below estimate

Most likely outcome As estimated

Example of a risk factor assessmentRate for office space for project team ($/m2/mth)

100%85% 150%

Office space rate uncertainty

Distribution sampledand multiplied intobase cost of office space

P10

P90

Total cost of office space

Rate uncertainty

Team size uncertainty

Base estimate costof office space $X

X

Multiply

Simulated costof office space

Add to othersimulatedproject costsin the model

Monte Carlo simulation

Sample Calculate Store

Risk factor example (partial)Small IT development project

Professional services $

Software scale

Team productivity

Professional services rates

Duration

Overhead rates ($/mth)

Overhead $

Market rates for installation licenses

Number of users

Design decision

Option 1

Option 2

Installation license cost $

Electrical & instrumentationSteel, mechanical & piping

ConcreteEarthworks

Risk factor example (partial)Mineral processing plant construction

Earthworks direct labour $

Earthworks quantity

Labour productivity

Labour rates

Lump sum valueDesign decision

Option 1

Option 2

Major equipment item cost $

Bulk material rates $/m3

Bulk material cost $

Summary

Some methods in common use today were selected when there was no alternative to pencil and paper They are not well suited to modelling project cost risk Simple methods became locked in to the way we work They have a role as part of the modelling toolkit but only part

Risk factor models were used over 40 years ago Take-up limited by access to computers Benefits identified at the outset are still relevant Rediscovered or redeveloped recently Simple, powerful and easy to understand

Questions and comments

Stephen Grey - grey@broadleaf.com.au

Reference to the World Bank paperhttp://documents.worldbank.org/curated/en/1970/01/6751137/risk-analysis-project-appraisal

Project Risk Management Guidelines: Managing Risk with ISO 31000 and IEC 62198

Dale Cooper, Pauline Bosnich, Stephen Grey, Grant Purdy, Geoffrey Raymond, Phil Walker, Mike Wood

ISBN: 978-1-118-82031-5