approximation and errors - caig lab - national chiao...

Approximation and Errors

Numerical Methods © Wen-Chieh Lin 2

Well-Posed Problems

Problem is well-posed if solution exits is unique depends continuously on problem data

Otherwise, problem is ill-posed


Remedy to ill-posed problem

Replace difficult problem by easier one havingsame or close enough solution nonlinear linear infinite finite differential algebraic complicate simple

Solution obtained may only approximate thatof the original problem


Sources of Approximation Before computation

Modeling Empirical measurements Previous computations

During computation Truncation or discretization Rounding

Accuracy of final result reflects all these Uncertainty in input may be amplified by problem Perturbations during computation may be amplified

by algorithmFrom Michael T. Heath’s slide


Example: ApproximationsComputing surface area of Earth using formula

involves several approximations

Earth is modeled as sphere, idealizing its trueshape

Value for radius is based on empiricalmeasurements and previous computations

Value forπrequires truncating infinite processValues for input data and results of arithmetic

operations are rounded in computer

24 rV

From Michael T. Heath’s slide


Error Definition

Absolute error: |approximate value –true value| Relative error: (absolute error)/(true value) True value is usually unknown, so we estimate or

bound error than compute it exactly Relative error often taken relative to approximate

value rather than (unknown) true value


ionapproxiamtcurrentionapproximatprevious-ionapproximatcurrent


Truncation ErrorDifference between true result and result

produced by given algorithm using exactarithmetic

Due to approximations such as truncatinginfinite series or terminating iterative sequencebefore convergence

!3!2!11

32 xxxex

4

32

!!3!2!11

n

nx

nxxxx

e

Truncation error


!11

xex

15.0 e1xe

5.15.0 e

%3.33%1005.1

15.1

ionapproxiamtcurrentionapproximatprevious-ionapproximatcurrent

4

32

!!3!2!11

n

nx

nxxxx

e


!11

xex

15.0 e1xe

5.15.0 e

%69.7%100625.1

1625.1

4

32

!!3!2!11

n

nx

nxxxx

e

!2!11

2xxex 625.15.0 e

%3.33


!11

xex

15.0 e1xe

5.15.0 e

%27.1

4

32

!!3!2!11

n

nx

nxxxx

e

!2!11

2xxex 625.15.0 e

%3.33

%69.7

!3!2!11

32 xxxex 645833333.15.0 e


Round-off Error

Difference between result produced by givenalgorithm using exact arithmetic and resultproduced by same algorithm using limitedprecision arithmetic

Due to inexact representation of real numbersand arithmetic operations upon them

Computational error = truncation error +round-off error


Propagated Error

An error in the succeeding steps of a processdue to an occurrence of an earlier error

Numerical Methods © Wen-Chieh Lin 13calcx

Forward and Backward Error

Suppose we want to compute y=f(x), where f is a realvalue function, but obtain approximate value

Forward error

Backward error

calcy

)(xfyexact

x

)(1calccalc yfx calcy

exacty

exactcalcfwd yyE

xxE calcbackw

Numerical Methods © Wen-Chieh Lin 14x

Forward and Backward Error

Suppose we want to compute y=f(x), where f is a realvalue function, but obtain approximate value

Forward error

Backward error

y

)(xfy

x

)(ˆ 1 yfx y

y

yyy ˆ

xxx ˆ


Backward Error Analysis Idea: approximate solution is exact solution to

modified problemHow much must original problem change to

give result actually obtained?How much data error in input would explain

all error in computed result?Approximate solution is good if it is exact

solution to near problem



Example: Backward Error Analysis

Approximating cosine function by truncatingTaylor series after two terms gives

Forward error is given by

Backward error is given by , where

21)(ˆ 2xxfy

)cos(21)()(ˆ 2 xxxfxfyy

)arccos()(ˆ 1 yyfx

xx


Example (cont.)

For x=1,

Forward error:Backward error:

5.0211)1(ˆ 2 fy

5403.0)1cos()1( fy

0472.1)5.0arccos()arccos(ˆ yx

0403.05403.05.0ˆ yyy

0472.010472.1ˆ xxx


Sensitivity and Conditioning Problem is insensitive or well-conditioned, if relative

change in input causes similar change in solution Problem is sensitive or ill-conditioned, if relative

change in solution can be much larger than that ininput

Condition number

xxxxfxfxf

)ˆ()()]()([

|datainputinchagerelative||solutioninchangerelative|

cond


Sensitivity and Conditioning (cont.)

Problem is sensitive or ill-conditioned, ifcond >> 1

A well-posed problem can be ill-conditioned

Computational algorithm should not makesensitivity worse


Condition Number Condition number is amplification factor relating

relative forward error to relative backward error

Condition number is usually not known exactly andmay vary with input, so rough estimate or upperbound is used

xxyy

xxxxfxfxf

)ˆ()()]()([

cond


)()(')()('

)()()]()([

)ˆ()()]()([

|datainputinchagerelative||solutioninchangerelative|

cond

xfxxf

xhxfxhf

xxhxxfxfhxf

xxxxfxfxf

Example: Evaluating Function Evaluating function f for approximate input

)(')()( xhfxfhxf

hxx

hxx


Example: Sensitivity Tangent function is sensitive for arguments

near π/2

For

51058058.1)57079.1tan(

41058058.6)57078.1tan(

51048275.2cond,57079.1 x


Stability Algorithm is stable if result produced is relatively

insensitive to perturbations during computation

Stability of algorithm is analogous to conditioning ofproblems

From viewpoint of backward error analysis,algorithm is stable if result produced is exact solutionto nearby problem

For stable algorithm, effect of computational error isworse than effect of small data error in input



Accuracy Accuracy: closeness of computed solution to true

solution of problem

Stability alone does not guarantee accurate results

Accuracy depends on conditioning of problem as wellas the stability of algorithm

Inaccuracy can result from applying stable algorithm to ill-conditioned problem unstable algorithm to well-conditioned problem

Applying stable algorithm to well-conditionedproblem yields accurate solution



Floating-Point Arithmetic

Floating-point number is represented by sign fraction (mantissa) exponent

Eppddd

x 2)222

1( 11

221

UELpidi and,1,,1,10where


Machine Numbers

Not all real numbers exactly representable;those that are are called machine numbers

Machine numbers are unequally spaced

If a real number is not exactly repersentable,then it is approximated by a nearby machinenumber causing round-off error


Machine Precision

Accuracy of floating-point systemcharacterized by unit round-off (or machineprecision or machine epsilon)

Smallest number such that

If1)1( epsfloat

)1(

)(1

eps


Machine Precision



If1)1( epsfloat

1)1(

)(1

eps


Machine Precision



If1)1( epsfloat

1)1(

1)(1

eps


Summary

Well-posed problemComputational error

truncation error round-off error

Sensitivity (conditioning) of problemCondition number

Stability of algorithm

approximation and errors - caig lab - national chiao...

Documents