1 standards and calibration laboratory, scl evaluation of measurement uncertainties using the monte...

1

Standards and Calibration Laboratory, SCL

Evaluation of Measurement Uncertainties Using the

Monte Carlo Method

Speaker: Chung Yin, PoonStandards and Calibration Laboratory (SCL)The Government of the Hong Kong Special

Administrative Region

2

Standards and Calibration Laboratory, SCL

GUM Uncertainty Framework (GUF)

“Propagation of Uncertainties”Measurement Model: Y= f(X1, X2, XN)Estimate xi of the input quantities Xi

Determine u(xi) associated with each estimate xi and its degrees of freedom

Estimate y = f(xi) of YCalculate the sensitivity coefficient of each xi at Xi = xi

Calculate u(y)Calculate the effective degrees of freedom veff and coverage factor k

with coverage probability pCalculate the coverage interval: yku(y)

3

Standards and Calibration Laboratory, SCL

GUM Uncertainty Framework (GUF)

Problems:The contributory uncertainties are not of

approximately the same magnitudeDifficult to provide the partial derivatives of the

modelThe PDF for output quantity is not a Gaussian

distribution or a scaled and shifted t-distribution

Y = f (X) y, u(y)

x1, u(x1)

x2, u(x2)

xN, u(xN)

4

Standards and Calibration Laboratory, SCL

Monte Carlo Method (MCM)“Propagation of Distributions”Measurement Model: Y= f(X1, X2, XN)Assign probability density function (PDF) to each XSelect M for the number of Monte Carlo trialsGenerate M vectors by sampling from the PDF of each X

(x1,1, x1,2, x1,M) (xN,1, xN,2, xN,M) Calculate M model values y = (f(x1,1, xN,1), f(x1,M, xN,M))Estimate y of Y and associated standard uncertainty u(y)Calculate the interval [ylow,yhigh] for Y with corresponding

coverage probability p

5

Standards and Calibration Laboratory, SCL

Monte Carlo Method (MCM)

Y = f (X)

gY(h)

gX1(x1)

gX2(x2)

gXN(xN)

6

Standards and Calibration Laboratory, SCL

Operation Modes For MCM

• There are three modes of operations– Fixed-Number-of-Trials Mode– Adaptive Mode– Approximated Adaptive (or Histogram) Mode

7

Standards and Calibration Laboratory, SCL

Adaptive Monte Carlo ProcedureSet number of significant digit

ndig (usually = 2)

Set M = max (100/(1-p), 104)

h = 1

Perform MCM trial

y(h)=(y1(h),y2

(h),...yM(h))

Calculate u(y(h)), ylow(h), yhigh

(h)

h > 1 ?

h = h + 1

Calculate sy, su(y), syhigh, sylow

Use h ́M model values calculate u(y) and then d

Any 2sy, 2su(y), 2sylow or 2syhigh > d/5?

Use h ́M model values calculate y,

u(y), ylow, yhighN

Y

Y

N

8

Standards and Calibration Laboratory, SCL

Validation of GUF

• Calculate: dlow = y – Up – ylow and

dhigh = y + Up – yhigh

• If both differences are not larger than , then the GUF is validated.

9

Standards and Calibration Laboratory, SCL

Histogram Procedure

• If the numerical tolerance is small, the value of M required would be larger. This may causes efficiency problems for some computers

• Experiences show that a very precise measurement will require a M of up to 107

• Using histogram to approximate the PDF

10

Standards and Calibration Laboratory, SCL

Histogram Procedure

1. Build the initial histogram for y withBin = 100,000

2. Continue generate the model Update y and u(y) for each iteration Check stabilization. (Same as the adaptive

procedure, i.e. check the four s values) Update the histogram Store the outliers (i.e. those values beyond the

boundaries of the histogram)

11

Standards and Calibration Laboratory, SCL

Histogram Procedure

4. When stabilized: Build complete histogram to include the

outliers Transform the histogram to a

distribution function Use this discrete approximation to

calculate the coverage interval

12

Standards and Calibration Laboratory, SCL

Determine Coverage Intervals

• By Inverse linear interpolation [Annex D.5 to D.8 of GS1]

y(r)

ylo

w

y(r+

1)

pr

pr+1

a

y

p GY(h)

y(s)

yh

igh

y(s+

1)

y

p GY(h)

ps+1

ps

p+a

13

Standards and Calibration Laboratory, SCL

Shortest Coverage Interval

• Repeat the method to determine a large number of intervals corresponding to(, p+) and find the minimum value.E.g. = 0 to 0.05 for 95 % coverage interval.

• The precision level is related to the incremental step of in the search.

• The step uses in this software is 0.0001, i.e. total 501 steps.

14

Standards and Calibration Laboratory, SCL

MCM Software

MCM Code Generator

(VB Codes)

MATLAB Script File

MCM Engine(MATLAB Codes)

- Perform MCM Procedure-GUF Validation

-Graphical Output of Results

15

Standards and Calibration Laboratory, SCL

GUI of the MCM Code Generator

16

Standards and Calibration Laboratory, SCL

Results for example 9.4.3.2 of GS1

• PDF for the y values in histogram• GUF Gaussian/t-distribution• Coverage Intervals• MCM and GUF results for y, u(y), ylow and yhigh

• GUF validation result• Number of MCM trials

17

Standards and Calibration Laboratory, SCL

ExampleCalibration of a 10 V Zener Voltage Reference using

Josephson Array Voltage Standard

• Measurement Model:• PDF parameters input to the software:

ranmOLJ

VVVVK

fny

Input Quantity PDF ParameterSymbol Description PDF /

Constant v a b

n Quantum (Step) Number

constant 63968

f Frequency N(, 2) 75.6 GHz 5.13 Hz

KJ Josephson Constant

constant 483597.9 GHz/V

VL Leakage R(a,b) -5 nV 5 nV

VO Offset R(a,b) -0.1 V 0.1 V

Vm Null Voltage R(a,b) 3.722 V 3.712 V 3.732 V

Vran Random Noise tv(, 2) 0 V 30 nV 39

18

Standards and Calibration Laboratory, SCL

Parameters Input to the MCM Code Generator

19

Standards and Calibration Laboratory, SCL

Results

Method y (V) u(y) (nV) ylow (nV) yhigh (nV)GUF 10 67 -131 +131

MCM1(Fixed number)

10 66 -120 +120

MCM2(Adaptive)

10 66 -120 +120

MCM3(Histogram)

10 66 -120 +120

Method dlow

(nV)dhigh

(nV)GUF

validated?No. of Trials Computation Time

(s)

MCM1(Fixed number)

-11 +11 No 1,000,000 < 2

MCM2(Adaptive)

-11 +11 No 6,210,000 89

MCM3(Histogram)

-11 +11 No 6,270,000 8

Computer Configurations:Windows XP; MATLAB R2008b (version 7.7); CPU: Core Due T5600, 1.83 GHz, 2 GB Ram, 80 GB Harddisk

20

Standards and Calibration Laboratory, SCL

Thank You