Chapter 3

Differential Calculus

3.1 Limits of Functions

Definition of a Limit. If the value of the function y(x) gets arbitrarily close to ψ as x approaches the point ξ,then we say that the limit of the function as x approaches ξ is equal to ψ. This is written:

limx→ξ

y(x) = ψ

Now we make the notion of “arbitrarily close” precise. For any ǫ > 0 there exists a δ > 0 such that |y(x) − ψ| < ǫfor all 0 < |x− ξ| < δ. That is, there is an interval surrounding the point x = ξ for which the function is within ǫ ofψ. See Figure 3.1. Note that the interval surrounding x = ξ is a deleted neighborhood, that is it does not contain thepoint x = ξ. Thus the value of the function at x = ξ need not be equal to ψ for the limit to exist. Indeed the functionneed not even be defined at x = ξ.

To prove that a function has a limit at a point ξ we first bound |y(x)− ψ| in terms of δ for values of x satisfying0 < |x− ξ| < δ. Denote this upper bound by u(δ). Then for an arbitrary ǫ > 0, we determine a δ > 0 such that thethe upper bound u(δ) and hence |y(x)− ψ| is less than ǫ.

48

x

y

ψ+ε

ψ−ε

ξ−δ ξ+δ

Figure 3.1: The δ neighborhood of x = ξ such that |y(x)− ψ| < ǫ.

Example 3.1.1 Show thatlimx→1

x2 = 1.

Consider any ǫ > 0. We need to show that there exists a δ > 0 such that |x2 − 1| < ǫ for all |x − 1| < δ. First weobtain a bound on |x2 − 1|.

|x2 − 1| = |(x− 1)(x+ 1)|= |x− 1||x+ 1|< δ|x+ 1|= δ|(x− 1) + 2|< δ(δ + 2)

Now we choose a positive δ such that,δ(δ + 2) = ǫ.

We see thatδ =

√1 + ǫ− 1,

is positive and satisfies the criterion that |x2 − 1| < ǫ for all 0 < |x− 1| < δ. Thus the limit exists.

49

Example 3.1.2 Recall that the value of the function y(ξ) need not be equal to limx→ξ y(x) for the limit to exist. Weshow an example of this. Consider the function

y(x) =

{

1 for x ∈ Z,

0 for x 6∈ Z.

For what values of ξ does limx→ξ y(x) exist?First consider ξ 6∈ Z. Then there exists an open neighborhood a < ξ < b around ξ such that y(x) is identically zero

for x ∈ (a, b). Then trivially, limx→ξ y(x) = 0.Now consider ξ ∈ Z. Consider any ǫ > 0. Then if |x − ξ| < 1 then |y(x) − 0| = 0 < ǫ. Thus we see that

limx→ξ y(x) = 0.Thus, regardless of the value of ξ, limx→ξ y(x) = 0.

Left and Right Limits. With the notation limx→ξ+ y(x) we denote the right limit of y(x). This is the limit as xapproaches ξ from above. Mathematically: limx→ξ+ exists if for any ǫ > 0 there exists a δ > 0 such that |y(x)−ψ| < ǫfor all 0 < ξ − x < δ. The left limit limx→ξ− y(x) is defined analogously.

Example 3.1.3 Consider the function, sinx|x|

, defined for x 6= 0. (See Figure 3.2.) The left and right limits exist as xapproaches zero.

limx→0+

sin x

|x| = 1, limx→0−

sin x

|x| = −1

However the limit,

limx→0

sin x

|x| ,

does not exist.

Properties of Limits. Let limx→ξ

f(x) and limx→ξ

g(x) exist.

• limx→ξ

(af(x) + bg(x)) = a limx→ξ

f(x) + b limx→ξ

g(x).

50

Figure 3.2: Plot of sin(x)/|x|.

• limx→ξ

(f(x)g(x)) =

(

limx→ξ

f(x)

)(

limx→ξ

g(x)

)

.

• limx→ξ

(

f(x)

g(x)

)

=limx→ξ f(x)

limx→ξ g(x)if limx→ξ

g(x) 6= 0.

Example 3.1.4 We prove that if limx→ξ f(x) = φ and limx→ξ g(x) = γ exist then

limx→ξ

(f(x)g(x)) =

(

limx→ξ

f(x)

)(

limx→ξ

g(x)

)

.

Since the limit exists for f(x), we know that for all ǫ > 0 there exists δ > 0 such that |f(x)− φ| < ǫ for |x− ξ| < δ.Likewise for g(x). We seek to show that for all ǫ > 0 there exists δ > 0 such that |f(x)g(x)− φγ| < ǫ for |x− ξ| < δ.We proceed by writing |f(x)g(x)− φγ|, in terms of |f(x)− φ| and |g(x)− γ|, which we know how to bound.

|f(x)g(x)− φγ| = |f(x)(g(x)− γ) + (f(x)− φ)γ|≤ |f(x)||g(x)− γ|+ |f(x)− φ||γ|

If we choose a δ such that |f(x)||g(x) − γ| < ǫ/2 and |f(x) − φ||γ| < ǫ/2 then we will have the desired result:|f(x)g(x)−φγ| < ǫ. Trying to ensure that |f(x)||g(x)−γ| < ǫ/2 is hard because of the |f(x)| factor. We will replacethat factor with a constant. We want to write |f(x)−φ||γ| < ǫ/2 as |f(x)−φ| < ǫ/(2|γ|), but this is problematic forthe case γ = 0. We fix these two problems and then proceed. We choose δ1 such that |f(x)− φ| < 1 for |x− ξ| < δ1.

51

This gives us the desired form.

|f(x)g(x)− φγ| ≤ (|φ|+ 1)|g(x)− γ|+ |f(x)− φ|(|γ|+ 1), for |x− ξ| < δ1

Next we choose δ2 such that |g(x)−γ| < ǫ/(2(|φ|+1)) for |x−ξ| < δ2 and choose δ3 such that |f(x)−φ| < ǫ/(2(|γ|+1))for |x− ξ| < δ3. Let δ be the minimum of δ1, δ2 and δ3.

|f(x)g(x)− φγ| ≤ (|φ|+ 1)|g(x)− γ|+ |f(x)− φ|(|γ|+ 1) <ǫ

2+ǫ

2, for |x− ξ| < δ

|f(x)g(x)− φγ| < ǫ, for |x− ξ| < δ

We conclude that the limit of a product is the product of the limits.

limx→ξ

(f(x)g(x)) =

(

limx→ξ

f(x)

)(

limx→ξ

g(x)

)

= φγ.

52

Result 3.1.1 Definition of a Limit. The statement:

limx→ξ

y(x) = ψ

means that y(x) gets arbitrarily close to ψ as x approaches ξ. For any ǫ > 0 there exists aδ > 0 such that |y(x)− ψ| < ǫ for all x in the neighborhood 0 < |x− ξ| < δ. The left andright limits,

limx→ξ−

y(x) = ψ and limx→ξ+

y(x) = ψ

denote the limiting value as x approaches ξ respectively from below and above. The neigh-borhoods are respectively −δ < x− ξ < 0 and 0 < x− ξ < δ.Properties of Limits. Let lim

x→ξu(x) and lim

x→ξv(x) exist.

• limx→ξ

(au(x) + bv(x)) = a limx→ξ

u(x) + b limx→ξ

v(x).

• limx→ξ

(u(x)v(x)) =

(

limx→ξ

u(x)

)(

limx→ξ

v(x)

)

.

• limx→ξ

(

u(x)

v(x)

)

=limx→ξ u(x)

limx→ξ v(x)if lim

x→ξv(x) 6= 0.

3.2 Continuous Functions

Definition of Continuity. A function y(x) is said to be continuous at x = ξ if the value of the function isequal to its limit, that is, limx→ξ y(x) = y(ξ). Note that this one condition is actually the three conditions: y(ξ) is

53

defined, limx→ξ y(x) exists and limx→ξ y(x) = y(ξ). A function is continuous if it is continuous at each point in itsdomain. A function is continuous on the closed interval [a, b] if the function is continuous for each point x ∈ (a, b) andlimx→a+ y(x) = y(a) and limx→b

−

y(x) = y(b).

Discontinuous Functions. If a function is not continuous at a point it is called discontinuous at that point. Iflimx→ξ y(x) exists but is not equal to y(ξ), then the function has a removable discontinuity. It is thus named becausewe could define a continuous function

z(x) =

{

y(x) for x 6= ξ,

limx→ξ y(x) for x = ξ,

to remove the discontinuity. If both the left and right limit of a function at a point exist, but are not equal, then thefunction has a jump discontinuity at that point. If either the left or right limit of a function does not exist, then thefunction is said to have an infinite discontinuity at that point.

Example 3.2.1 sinxx

has a removable discontinuity at x = 0. The Heaviside function,

H(x) =

0 for x < 0,

1/2 for x = 0,

1 for x > 0,

has a jump discontinuity at x = 0. 1x

has an infinite discontinuity at x = 0. See Figure 3.3.

Properties of Continuous Functions.

Arithmetic. If u(x) and v(x) are continuous at x = ξ then u(x)± v(x) and u(x)v(x) are continuous at x = ξ. u(x)v(x)

is continuous at x = ξ if v(ξ) 6= 0.

Function Composition. If u(x) is continuous at x = ξ and v(x) is continuous at x = µ = u(ξ) then u(v(x)) iscontinuous at x = ξ. The composition of continuous functions is a continuous function.

54

Figure 3.3: A Removable discontinuity, a Jump Discontinuity and an Infinite Discontinuity

Boundedness. A function which is continuous on a closed interval is bounded in that closed interval.

Nonzero in a Neighborhood. If y(ξ) 6= 0 then there exists a neighborhood (ξ − ǫ, ξ + ǫ), ǫ > 0 of the point ξ suchthat y(x) 6= 0 for x ∈ (ξ − ǫ, ξ + ǫ).

Intermediate Value Theorem. Let u(x) be continuous on [a, b]. If u(a) ≤ µ ≤ u(b) then there exists ξ ∈ [a, b] suchthat u(ξ) = µ. This is known as the intermediate value theorem. A corollary of this is that if u(a) and u(b) areof opposite sign then u(x) has at least one zero on the interval (a, b).

Maxima and Minima. If u(x) is continuous on [a, b] then u(x) has a maximum and a minimum on [a, b]. That is, thereis at least one point ξ ∈ [a, b] such that u(ξ) ≥ u(x) for all x ∈ [a, b] and there is at least one point ψ ∈ [a, b]such that u(ψ) ≤ u(x) for all x ∈ [a, b].

Piecewise Continuous Functions. A function is piecewise continuous on an interval if the function is bounded onthe interval and the interval can be divided into a finite number of intervals on each of which the function is continuous.For example, the greatest integer function, ⌊x⌋, is piecewise continuous. (⌊x⌋ is defined to the the greatest integer lessthan or equal to x.) See Figure 3.4 for graphs of two piecewise continuous functions.

Uniform Continuity. Consider a function f(x) that is continuous on an interval. This means that for any point ξin the interval and any positive ǫ there exists a δ > 0 such that |f(x)− f(ξ)| < ǫ for all 0 < |x− ξ| < δ. In general,this value of δ depends on both ξ and ǫ. If δ can be chosen so it is a function of ǫ alone and independent of ξ then

55

Figure 3.4: Piecewise Continuous Functions

the function is said to be uniformly continuous on the interval. A sufficient condition for uniform continuity is that thefunction is continuous on a closed interval.

3.3 The Derivative

Consider a function y(x) on the interval (x . . . x+∆x) for some ∆x > 0. We define the increment ∆y = y(x+∆x)−y(x). The average rate of change, (average velocity), of the function on the interval is ∆y

∆x. The average rate of change

is the slope of the secant line that passes through the points (x, y(x)) and (x+ ∆x, y(x+ ∆x)). See Figure 3.5.

y

x

∆y

∆x

Figure 3.5: The increments ∆x and ∆y.

56

If the slope of the secant line has a limit as ∆x approaches zero then we call this slope the derivative or instantaneousrate of change of the function at the point x. We denote the derivative by dy

dx, which is a nice notation as the derivative

is the limit of ∆y∆x

as ∆x→ 0.

dy

dx≡ lim

∆x→0

y(x+ ∆x)− y(x)

∆x.

∆x may approach zero from below or above. It is common to denote the derivative dydx

by ddxy, y′(x), y′ or Dy.

A function is said to be differentiable at a point if the derivative exists there. Note that differentiability impliescontinuity, but not vice versa.

Example 3.3.1 Consider the derivative of y(x) = x2 at the point x = 1.

y′(1) ≡ lim∆x→0

y(1 + ∆x)− y(1)

∆x

= lim∆x→0

(1 + ∆x)2 − 1

∆x= lim

∆x→0(2 + ∆x)

= 2

Figure 3.6 shows the secant lines approaching the tangent line as ∆x approaches zero from above and below.

Example 3.3.2 We can compute the derivative of y(x) = x2 at an arbitrary point x.

d

dx

[

x2]

= lim∆x→0

(x+ ∆x)2 − x2

∆x= lim

∆x→0(2x+ ∆x)

= 2x

57

0.5 1 1.5 2

0.5

1

1.5

2

2.5

3

3.5

4

0.5 1 1.5 2

0.5

1

1.5

2

2.5

3

3.5

4

Figure 3.6: Secant lines and the tangent to x2 at x = 1.

Properties. Let u(x) and v(x) be differentiable. Let a and b be constants. Some fundamental properties ofderivatives are:

d

dx(au+ bv) = a

du

dx+ b

dv

dxLinearity

d

dx(uv) =

du

dxv + u

dv

dxProduct Rule

d

dx

(u

v

)

=v du

dx− udv

dx

v2Quotient Rule

d

dx(ua) = aua−1 du

dxPower Rule

d

dx(u(v(x))) =

du

dv

dv

dx= u′(v(x))v′(x) Chain Rule

These can be proved by using the definition of differentiation.

58

Example 3.3.3 Prove the quotient rule for derivatives.

d

dx

(u

v

)

= lim∆x→0

u(x+∆x)v(x+∆x)

− u(x)v(x)

∆x

= lim∆x→0

u(x+ ∆x)v(x)− u(x)v(x+ ∆x)

∆xv(x)v(x+ ∆x)

= lim∆x→0

u(x+ ∆x)v(x)− u(x)v(x)− u(x)v(x+ ∆x) + u(x)v(x)

∆xv(x)v(x)

= lim∆x→0

(u(x+ ∆x)− u(x))v(x)− u(x)(v(x+ ∆x)− v(x))

∆xv2(x)

=lim∆x→0

u(x+∆x)−u(x)∆x

v(x)− u(x) lim∆x→0v(x+∆x)−v(x)

∆x

v2(x)

=v du

dx− udv

dx

v2

59

Trigonometric Functions. Some derivatives of trigonometric functions are:

d

dxsin x = cosx

d

dxarcsinx =

1

(1− x2)1/2

d

dxcosx = − sin x

d

dxarccosx =

−1

(1− x2)1/2

d

dxtanx =

1

cos2 x

d

dxarctanx =

1

1 + x2

d

dxex = ex

d

dxlnx =

1

xd

dxsinh x = coshx

d

dxarcsinhx =

1

(x2 + 1)1/2

d

dxcoshx = sinhx

d

dxarccoshx =

1

(x2 − 1)1/2

d

dxtanhx =

1

cosh2 x

d

dxarctanhx =

1

1− x2

Example 3.3.4 We can evaluate the derivative of xx by using the identity ab = eb ln a.

d

dxxx =

d

dxex lnx

= ex lnx d

dx(x lnx)

= xx(1 · lnx+ x1

x)

= xx(1 + lnx)

Inverse Functions. If we have a function y(x), we can consider x as a function of y, x(y). For example, ify(x) = 8x3 then x(y) = 3

√y/2; if y(x) = x+2

x+1then x(y) = 2−y

y−1. The derivative of an inverse function is

d

dyx(y) =

1dydx

.

60

Example 3.3.5 The inverse function of y(x) = ex is x(y) = ln y. We can obtain the derivative of the logarithm fromthe derivative of the exponential. The derivative of the exponential is

dy

dx= ex .

Thus the derivative of the logarithm is

d

dyln y =

d

dyx(y) =

1dydx

=1

ex=

1

y.

3.4 Implicit Differentiation

An explicitly defined function has the form y = f(x). A implicitly defined function has the form f(x, y) = 0. A fewexamples of implicit functions are x2 + y2− 1 = 0 and x+ y+sin(xy) = 0. Often it is not possible to write an implicitequation in explicit form. This is true of the latter example above. One can calculate the derivative of y(x) in termsof x and y even when y(x) is defined by an implicit equation.

Example 3.4.1 Consider the implicit equation

x2 − xy − y2 = 1.

This implicit equation can be solved for the dependent variable.

y(x) =1

2

(

−x±√

5x2 − 4)

.

We can differentiate this expression to obtain

y′ =1

2

(

−1± 5x√5x2 − 4

)

.

One can obtain the same result without first solving for y. If we differentiate the implicit equation, we obtain

2x− y − xdy

dx− 2y

dy

dx= 0.

61

We can solve this equation for dydx

.

dy

dx=

2x− y

x+ 2y

We can differentiate this expression to obtain the second derivative of y.

d2y

dx2=

(x+ 2y)(2− y′)− (2x− y)(1 + 2y′)

(x+ 2y)2

=5(y − xy′)

(x+ 2y)2

Substitute in the expression for y′.

= −10(x2 − xy − y2)

(x+ 2y)2

Use the original implicit equation.

= − 10

(x+ 2y)2

3.5 Maxima and Minima

A differentiable function is increasing where f ′(x) > 0, decreasing where f ′(x) < 0 and stationary where f ′(x) = 0.

A function f(x) has a relative maxima at a point x = ξ if there exists a neighborhood around ξ such that f(x) ≤ f(ξ)for x ∈ (x − δ, x + δ), δ > 0. The relative minima is defined analogously. Note that this definition does not requirethat the function be differentiable, or even continuous. We refer to relative maxima and minima collectively are relativeextrema.

62

Relative Extrema and Stationary Points. If f(x) is differentiable and f(ξ) is a relative extrema then x = ξis a stationary point, f ′(ξ) = 0. We can prove this using left and right limits. Assume that f(ξ) is a relative maxima.Then there is a neighborhood (x− δ, x + δ), δ > 0 for which f(x) ≤ f(ξ). Since f(x) is differentiable the derivativeat x = ξ,

f ′(ξ) = lim∆x→0

f(ξ + ∆x)− f(ξ)

∆x,

exists. This in turn means that the left and right limits exist and are equal. Since f(x) ≤ f(ξ) for ξ − δ < x < ξ theleft limit is non-positive,

f ′(ξ) = lim∆x→0−

f(ξ + ∆x)− f(ξ)

∆x≤ 0.

Since f(x) ≤ f(ξ) for ξ < x < ξ + δ the right limit is nonnegative,

f ′(ξ) = lim∆x→0+

f(ξ + ∆x)− f(ξ)

∆x≥ 0.

Thus we have 0 ≤ f ′(ξ) ≤ 0 which implies that f ′(ξ) = 0.

It is not true that all stationary points are relative extrema. That is, f ′(ξ) = 0 does not imply that x = ξ is anextrema. Consider the function f(x) = x3. x = 0 is a stationary point since f ′(x) = x2, f ′(0) = 0. However, x = 0 isneither a relative maxima nor a relative minima.

It is also not true that all relative extrema are stationary points. Consider the function f(x) = |x|. The point x = 0is a relative minima, but the derivative at that point is undefined.

First Derivative Test. Let f(x) be differentiable and f ′(ξ) = 0.

• If f ′(x) changes sign from positive to negative as we pass through x = ξ then the point is a relative maxima.

• If f ′(x) changes sign from negative to positive as we pass through x = ξ then the point is a relative minima.

63

• If f ′(x) is not identically zero in a neighborhood of x = ξ and it does not change sign as we pass through thepoint then x = ξ is not a relative extrema.

Example 3.5.1 Consider y = x2 and the point x = 0. The function is differentiable. The derivative, y′ = 2x, vanishesat x = 0. Since y′(x) is negative for x < 0 and positive for x > 0, the point x = 0 is a relative minima. See Figure 3.7.

Example 3.5.2 Consider y = cosx and the point x = 0. The function is differentiable. The derivative, y′ = − sin xis positive for −π < x < 0 and negative for 0 < x < π. Since the sign of y′ goes from positive to negative, x = 0 is arelative maxima. See Figure 3.7.

Example 3.5.3 Consider y = x3 and the point x = 0. The function is differentiable. The derivative, y′ = 3x2 ispositive for x < 0 and positive for 0 < x. Since y′ is not identically zero and the sign of y′ does not change, x = 0 isnot a relative extrema. See Figure 3.7.

Figure 3.7: Graphs of x2, cosx and x3.

Concavity. If the portion of a curve in some neighborhood of a point lies above the tangent line through that point,the curve is said to be concave upward. If it lies below the tangent it is concave downward. If a function is twicedifferentiable then f ′′(x) > 0 where it is concave upward and f ′′(x) < 0 where it is concave downward. Note thatf ′′(x) > 0 is a sufficient, but not a necessary condition for a curve to be concave upward at a point. A curve may beconcave upward at a point where the second derivative vanishes. A point where the curve changes concavity is calleda point of inflection. At such a point the second derivative vanishes, f ′′(x) = 0. For twice continuously differentiable

64

functions, f ′′(x) = 0 is a necessary but not a sufficient condition for an inflection point. The second derivative mayvanish at places which are not inflection points. See Figure 3.8.

Figure 3.8: Concave Upward, Concave Downward and an Inflection Point.

Second Derivative Test. Let f(x) be twice differentiable and let x = ξ be a stationary point, f ′(ξ) = 0.

• If f ′′(ξ) < 0 then the point is a relative maxima.

• If f ′′(ξ) > 0 then the point is a relative minima.

• If f ′′(ξ) = 0 then the test fails.

Example 3.5.4 Consider the function f(x) = cosx and the point x = 0. The derivatives of the function aref ′(x) = − sin x, f ′′(x) = − cosx. The point x = 0 is a stationary point, f ′(0) = − sin(0) = 0. Since the secondderivative is negative there, f ′′(0) = − cos(0) = −1, the point is a relative maxima.

Example 3.5.5 Consider the function f(x) = x4 and the point x = 0. The derivatives of the function are f ′(x) = 4x3,f ′′(x) = 12x2. The point x = 0 is a stationary point. Since the second derivative also vanishes at that point thesecond derivative test fails. One must use the first derivative test to determine that x = 0 is a relative minima.

65

3.6 Mean Value Theorems

Rolle’s Theorem. If f(x) is continuous in [a, b], differentiable in (a, b) and f(a) = f(b) = 0 then there exists apoint ξ ∈ (a, b) such that f ′(ξ) = 0. See Figure 3.9.

Figure 3.9: Rolle’s Theorem.

To prove this we consider two cases. First we have the trivial case that f(x) ≡ 0. If f(x) is not identically zerothen continuity implies that it must have a nonzero relative maxima or minima in (a, b). Let x = ξ be one of theserelative extrema. Since f(x) is differentiable, x = ξ must be a stationary point, f ′(ξ) = 0.

Theorem of the Mean. If f(x) is continuous in [a, b] and differentiable in (a, b) then there exists a point x = ξsuch that

f ′(ξ) =f(b)− f(a)

b− a.

That is, there is a point where the instantaneous velocity is equal to the average velocity on the interval.We prove this theorem by applying Rolle’s theorem. Consider the new function

g(x) = f(x)− f(a)− f(b)− f(a)

b− a(x− a)

Note that g(a) = g(b) = 0, so it satisfies the conditions of Rolle’s theorem. There is a point x = ξ such that g′(ξ) = 0.We differentiate the expression for g(x) and substitute in x = ξ to obtain the result.

g′(x) = f ′(x)− f(b)− f(a)

b− a

66

Figure 3.10: Theorem of the Mean.

g′(ξ) = f ′(ξ)− f(b)− f(a)

b− a= 0

f ′(ξ) =f(b)− f(a)

b− a

Generalized Theorem of the Mean. If f(x) and g(x) are continuous in [a, b] and differentiable in (a, b), thenthere exists a point x = ξ such that

f ′(ξ)

g′(ξ)=f(b)− f(a)

g(b)− g(a).

We have assumed that g(a) 6= g(b) so that the denominator does not vanish and that f ′(x) and g′(x) are notsimultaneously zero which would produce an indeterminate form. Note that this theorem reduces to the regulartheorem of the mean when g(x) = x. The proof of the theorem is similar to that for the theorem of the mean.

Taylor’s Theorem of the Mean. If f(x) is n + 1 times continuously differentiable in (a, b) then there exists apoint x = ξ ∈ (a, b) such that

f(b) = f(a) + (b− a)f ′(a) +(b− a)2

2!f ′′(a) + · · ·+ (b− a)n

n!f (n)(a) +

(b− a)n+1

(n+ 1)!f (n+1)(ξ). (3.1)

For the case n = 0, the formula isf(b) = f(a) + (b− a)f ′(ξ),

67

which is just a rearrangement of the terms in the theorem of the mean,

f ′(ξ) =f(b)− f(a)

b− a.

3.6.1 Application: Using Taylor’s Theorem to Approximate Functions.

One can use Taylor’s theorem to approximate functions with polynomials. Consider an infinitely differentiable functionf(x) and a point x = a. Substituting x for b into Equation 3.1 we obtain,

f(x) = f(a) + (x− a)f ′(a) +(x− a)2

2!f ′′(a) + · · ·+ (x− a)n

n!f (n)(a) +

(x− a)n+1

(n+ 1)!f (n+1)(ξ).

If the last term in the sum is small then we can approximate our function with an nth order polynomial.

f(x) ≈ f(a) + (x− a)f ′(a) +(x− a)2

2!f ′′(a) + · · ·+ (x− a)n

n!f (n)(a)

The last term in Equation 3.6.1 is called the remainder or the error term,

Rn =(x− a)n+1

(n+ 1)!f (n+1)(ξ).

Since the function is infinitely differentiable, f (n+1)(ξ) exists and is bounded. Therefore we note that the error mustvanish as x→ 0 because of the (x− a)n+1 factor. We therefore suspect that our approximation would be a good oneif x is close to a. Also note that n! eventually grows faster than (x− a)n,

limn→∞

(x− a)n

n!= 0.

So if the derivative term, f (n+1)(ξ), does not grow to quickly, the error for a certain value of x will get smaller withincreasing n and the polynomial will become a better approximation of the function. (It is also possible that thederivative factor grows very quickly and the approximation gets worse with increasing n.)

68

Example 3.6.1 Consider the function f(x) = ex. We want a polynomial approximation of this function near the pointx = 0. Since the derivative of ex is ex, the value of all the derivatives at x = 0 is f (n)(0) = e0 = 1. Taylor’s theoremthus states that

ex = 1 + x+x2

2!+x3

3!+ · · ·+ xn

n!+

xn+1

(n+ 1)!eξ,

for some ξ ∈ (0, x). The first few polynomial approximations of the exponent about the point x = 0 are

f1(x) = 1

f2(x) = 1 + x

f3(x) = 1 + x+x2

2

f4(x) = 1 + x+x2

2+x3

6

The four approximations are graphed in Figure 3.11.

-1 -0.5 0.5 1

0.5

1

1.5

2

2.5

-1 -0.5 0.5 1

0.5

1

1.5

2

2.5

-1 -0.5 0.5 1

0.5

1

1.5

2

2.5

-1 -0.5 0.5 1

0.5

1

1.5

2

2.5

Figure 3.11: Four Finite Taylor Series Approximations of ex

Note that for the range of x we are looking at, the approximations become more accurate as the number of termsincreases.

Example 3.6.2 Consider the function f(x) = cosx. We want a polynomial approximation of this function near the

69

point x = 0. The first few derivatives of f are

f(x) = cosx

f ′(x) = − sin x

f ′′(x) = − cosx

f ′′′(x) = sinx

f (4)(x) = cosx

It’s easy to pick out the pattern here,

f (n)(x) =

{

(−1)n/2 cosx for even n,

(−1)(n+1)/2 sin x for odd n.

Since cos(0) = 1 and sin(0) = 0 the n-term approximation of the cosine is,

cosx = 1− x2

2!+x4

4!− x6

6!+ · · ·+ (−1)2(n−1) x2(n−1)

(2(n− 1))!+

x2n

(2n)!cos ξ.

Here are graphs of the one, two, three and four term approximations.

-3 -2 -1 1 2 3

-1

-0.5

0.5

1

-3 -2 -1 1 2 3

-1

-0.5

0.5

1

-3 -2 -1 1 2 3

-1

-0.5

0.5

1

-3 -2 -1 1 2 3

-1

-0.5

0.5

1

Figure 3.12: Taylor Series Approximations of cosx

Note that for the range of x we are looking at, the approximations become more accurate as the number of termsincreases. Consider the ten term approximation of the cosine about x = 0,

cosx = 1− x2

2!+x4

4!− · · · − x18

18!+x20

20!cos ξ.

70

Note that for any value of ξ, | cos ξ| ≤ 1. Therefore the absolute value of the error term satisfies,

|R| =∣

∣

∣

∣

x20

20!cos ξ

∣

∣

∣

∣

≤ |x|20

20!.

x20/20! is plotted in Figure 3.13.

2 4 6 8 10

0.2

0.4

0.6

0.8

1

Figure 3.13: Plot of x20/20!.

Note that the error is very small for x < 6, fairly small but non-negligible for x ≈ 7 and large for x > 8. The tenterm approximation of the cosine, plotted below, behaves just we would predict.

The error is very small until it becomes non-negligible at x ≈ 7 and large at x ≈ 8.

Example 3.6.3 Consider the function f(x) = lnx. We want a polynomial approximation of this function near the

71

-10 -5 5 10

-2

-1.5

-1

-0.5

0.5

1

Figure 3.14: Ten Term Taylor Series Approximation of cosx

point x = 1. The first few derivatives of f are

f(x) = lnx

f ′(x) =1

x

f ′′(x) = − 1

x2

f ′′′(x) =2

x3

f (4)(x) = − 3

x4

The derivatives evaluated at x = 1 are

f(0) = 0, f (n)(0) = (−1)n−1(n− 1)!, for n ≥ 1.

By Taylor’s theorem of the mean we have,

lnx = (x− 1)− (x− 1)2

2+

(x− 1)3

3− (x− 1)4

4+ · · ·+ (−1)n−1 (x− 1)n

n+ (−1)n

(x− 1)n+1

n+ 1

1

ξn+1.

72

0.5 1 1.5 2 2.5 3

-6-5-4-3-2-1

12

0.5 1 1.5 2 2.5 3

-6-5-4-3-2-1

12

0.5 1 1.5 2 2.5 3

-6-5-4-3-2-1

12

0.5 1 1.5 2 2.5

-6-5-4-3-2-1

12

Figure 3.15: The 2, 4, 10 and 50 Term Approximations of lnx

Below are plots of the 2, 4, 10 and 50 term approximations.Note that the approximation gets better on the interval (0, 2) and worse outside this interval as the number of terms

increases. The Taylor series converges to lnx only on this interval.

3.6.2 Application: Finite Difference Schemes

Example 3.6.4 Suppose you sample a function at the discrete points n∆x, n ∈ Z. In Figure 3.16 we sample thefunction f(x) = sinx on the interval [−4, 4] with ∆x = 1/4 and plot the data points.

-4 -2 2 4

-1

-0.5

0.5

1

Figure 3.16: Sampling of sinx

We wish to approximate the derivative of the function on the grid points using only the value of the function on

73

those discrete points. From the definition of the derivative, one is lead to the formula

f ′(x) ≈ f(x+ ∆x)− f(x)

∆x. (3.2)

Taylor’s theorem states that

f(x+ ∆x) = f(x) + ∆xf ′(x) +∆x2

2f ′′(ξ).

Substituting this expression into our formula for approximating the derivative we obtain

f(x+ ∆x)− f(x)

∆x=f(x) + ∆xf ′(x) + ∆x2

2f ′′(ξ)− f(x)

∆x= f ′(x) +

∆x

2f ′′(ξ).

Thus we see that the error in our approximation of the first derivative is ∆x2f ′′(ξ). Since the error has a linear factor

of ∆x, we call this a first order accurate method. Equation 3.2 is called the forward difference scheme for calculatingthe first derivative. Figure 3.17 shows a plot of the value of this scheme for the function f(x) = sinx and ∆x = 1/4.The first derivative of the function f ′(x) = cosx is shown for comparison.

-4 -2 2 4

-1

-0.5

0.5

1

Figure 3.17: The Forward Difference Scheme Approximation of the Derivative

Another scheme for approximating the first derivative is the centered difference scheme,

f ′(x) ≈ f(x+ ∆x)− f(x−∆x)

2∆x.

74

Expanding the numerator using Taylor’s theorem,

f(x+ ∆x)− f(x−∆x)

2∆x

=f(x) + ∆xf ′(x) + ∆x2

2f ′′(x) + ∆x3

6f ′′′(ξ)− f(x) + ∆xf ′(x)− ∆x2

2f ′′(x) + ∆x3

6f ′′′(ψ)

2∆x

= f ′(x) +∆x2

12(f ′′′(ξ) + f ′′′(ψ)).

The error in the approximation is quadratic in ∆x. Therefore this is a second order accurate scheme. Below is a plotof the derivative of the function and the value of this scheme for the function f(x) = sinx and ∆x = 1/4.

-4 -2 2 4

-1

-0.5

0.5

1

Figure 3.18: Centered Difference Scheme Approximation of the Derivative

Notice how the centered difference scheme gives a better approximation of the derivative than the forward differencescheme.

3.7 L’Hospital’s Rule

Some singularities are easy to diagnose. Consider the function cosxx

at the point x = 0. The function evaluatesto 1

0and is thus discontinuous at that point. Since the numerator and denominator are continuous functions and the

75

denominator vanishes while the numerator does not, the left and right limits as x→ 0 do not exist. Thus the functionhas an infinite discontinuity at the point x = 0. More generally, a function which is composed of continuous functionsand evaluates to a

0at a point where a 6= 0 must have an infinite discontinuity there.

Other singularities require more analysis to diagnose. Consider the functions sinxx

, sinx|x|

and sinx1−cosx

at the point x = 0.

All three functions evaluate to 00

at that point, but have different kinds of singularities. The first has a removablediscontinuity, the second has a finite discontinuity and the third has an infinite discontinuity. See Figure 3.19.

Figure 3.19: The functions sinxx

, sinx|x|

and sinx1−cosx

.

An expression that evaluates to 00, ∞∞

, 0 ·∞, ∞−∞, 1∞, 00 or∞0 is called an indeterminate. A function f(x) whichis indeterminate at the point x = ξ is singular at that point. The singularity may be a removable discontinuity, a finitediscontinuity or an infinite discontinuity depending on the behavior of the function around that point. If limx→ξ f(x)exists, then the function has a removable discontinuity. If the limit does not exist, but the left and right limits do exist,then the function has a finite discontinuity. If either the left or right limit does not exist then the function has an infinitediscontinuity.

76

L’Hospital’s Rule. Let f(x) and g(x) be differentiable and f(ξ) = g(ξ) = 0. Further, let g(x) be nonzero in adeleted neighborhood of x = ξ, (g(x) 6= 0 for x ∈ 0 < |x− ξ| < δ). Then

limx→ξ

f(x)

g(x)= lim

x→ξ

f ′(x)

g′(x).

To prove this, we note that f(ξ) = g(ξ) = 0 and apply the generalized theorem of the mean. Note that

f(x)

g(x)=f(x)− f(ξ)

g(x)− g(ξ)=f ′(ψ)

g′(ψ)

for some ψ between ξ and x. Thus

limx→ξ

f(x)

g(x)= lim

ψ→ξ

f ′(ψ)

g′(ψ)= lim

x→ξ

f ′(x)

g′(x)

provided that the limits exist.L’Hospital’s Rule is also applicable when both functions tend to infinity instead of zero or when the limit point, ξ,

is at infinity. It is also valid for one-sided limits.L’Hospital’s rule is directly applicable to the indeterminate forms 0

0and ∞

∞.

Example 3.7.1 Consider the three functions sinxx

, sinx|x|

and sinx1−cosx

at the point x = 0.

limx→0

sin x

x= lim

x→0

cosx

1= 1

Thus sinxx

has a removable discontinuity at x = 0.

limx→0+

sin x

|x| = limx→0+

sin x

x= 1

limx→0−

sin x

|x| = limx→0−

sin x

−x = −1

77

Thus sinx|x|

has a finite discontinuity at x = 0.

limx→0

sin x

1− cosx= lim

x→0

cosx

sin x=

1

0= ∞

Thus sinx1−cosx

has an infinite discontinuity at x = 0.

Example 3.7.2 Let a and d be nonzero.

limx→∞

ax2 + bx+ c

dx2 + ex+ f= lim

x→∞

2ax+ b

2dx+ e

= limx→∞

2a

2d

=a

d

Example 3.7.3 Consider

limx→0

cosx− 1

x sin x.

This limit is an indeterminate of the form 00. Applying L’Hospital’s rule we see that limit is equal to

limx→0

− sin x

x cosx+ sinx.

This limit is again an indeterminate of the form 00. We apply L’Hospital’s rule again.

limx→0

− cosx

−x sin x+ 2 cosx= −1

2

78

Thus the value of the original limit is −12. We could also obtain this result by expanding the functions in Taylor series.

limx→0

cosx− 1

x sin x= lim

x→0

(

1− x2

2+ x4

24− · · ·

)

− 1

x(

x− x3

6+ x5

120− · · ·

)

= limx→0

−x2

2+ x4

24− · · ·

x2 − x4

6+ x6

120− · · ·

= limx→0

−12

+ x2

24− · · ·

1− x2

6+ x4

120− · · ·

= −1

2

We can apply L’Hospital’s Rule to the indeterminate forms 0 · ∞ and ∞ −∞ by rewriting the expression in adifferent form, (perhaps putting the expression over a common denominator). If at first you don’t succeed, try, tryagain. You may have to apply L’Hospital’s rule several times to evaluate a limit.

Example 3.7.4

limx→0

(

cotx− 1

x

)

= limx→0

x cosx− sin x

x sin x

= limx→0

cosx− x sin x− cosx

sin x+ x cosx

= limx→0

−x sin x

sin x+ x cosx

= limx→0

−x cosx− sin x

cosx+ cosx− x sin x

= 0

You can apply L’Hospital’s rule to the indeterminate forms 1∞, 00 or ∞0 by taking the logarithm of the expression.

79

Example 3.7.5 Consider the limit,limx→0

xx,

which gives us the indeterminate form 00. The logarithm of the expression is

ln(xx) = x lnx.

As x→ 0 we now have the indeterminate form 0 · ∞. By rewriting the expression, we can apply L’Hospital’s rule.

limx→0

lnx

1/x= lim

x→0

1/x

−1/x2

= limx→0

(−x)

= 0

Thus the original limit islimx→0

xx = e0 = 1.

80