Lecture Notes in Economic Mathematics

Weikai Chen

August 28, 2017

Contents

1 Preliminaries on Sets, Mappings, Relations, and Real Num-bers 31.1 Unions and intersections of sets . . . . . . . . . . . . . . . . . 31.2 Mappings between sets . . . . . . . . . . . . . . . . . . . . . . 41.3 Equivalence Relations* . . . . . . . . . . . . . . . . . . . . . . 51.4 The real numbers . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Basic Linear Algebra 82.1 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 Inner Products . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.4 Linear Transformations . . . . . . . . . . . . . . . . . . . . . . 132.5 The Four Fundamental Subspaces. . . . . . . . . . . . . . . . . 172.6 Determinant and Matrix Inversion. . . . . . . . . . . . . . . . 192.7 Eigenvectors and Eigenvalues* . . . . . . . . . . . . . . . . . . 22

3 Rudiments of Analysis 243.1 Convergence of Sequences . . . . . . . . . . . . . . . . . . . . 24

1

Introduction

These notes are prepared for the math review at Umass Amherst, summer2017. The review is based on the following textbook:

Chiang, Alpha and Wainwright, Kevin, Fundamental Methods ofMathematical Economics, (Fourth Edition) McGraw Hill, 2005.

This is also the required textbook for the fall semester course ECON 751(Mathematical Methods in Economics). We will cover the following sectionsin the first 8 chapters: 3.2-3.5, 4.3-4.6, 5.3-5.6, 6.4, 6.7, 7.4-7.6, 8.4, 8.5. Thestudents enrolled in ECON 751 during the fall will take an exam in the firstweek of the semester. This exam, containing questions based on the end ofchapter exercises for the above sections, will count for 20% of the final gradefor ECON 751.

Note: I will follow these lecture notes instead of the textbook. Some topicsin linear algebra and real analysis that are useful but not covered in thetextbook will be discussed in the lecture. Daily problem set will be assigned.

2

1 Preliminaries on Sets, Mappings, Relations,

and Real Numbers

In these preliminaries we set up the scene by describing some notions re-garding sets, mappings and relations that will be used throughout the notes.Also, we will discuss the axioms on real numbers, which is quite fundamentalbut not covered in the textbook.

1.1 Unions and intersections of sets

For a set A, an element x may be in A, denoted by x ∈ A, or not, byx /∈ A.We also call a member of A a point in A. Let A,B be sets, we call Ais a subset of B if each member of A is a member of B. That is,

A ⊆ B ⇔ ∀x ∈ A, x ∈ B.

We may also say A is contained in B or B contains A. Two sets A and Bare the same provided they have the same members, denoted by A = B. Wehave

A = B ⇔ A ⊆ B,B ⊆ A.

We say A is a proper subset of B if A ⊆ B but A 6= B.The union of A and B, denoted by A ∪ B, is the set of all points that

belongs either to A or to B; that is

A ∪B = {x|x ∈ A or x ∈ B}.

The intersection of A and B is defined as

A ∩B = {x|x ∈ A and x ∈ B}.

The complement of A in B, denoted by B ∼ A, is the set of all points inB that is not in A; that is

B ∼ A = {x|x ∈ B, x /∈ B}.

Let U be the universe set, then we often refer to U ∼ A simply as thecomplement of A, denoted by Ac.

The set that has no elements is called the empty-set, denoted by ∅, andthe set that has a single member is called a singleton set. Given a set X,the power set of X is the set of all subsets of X, denoted by P(X) or 2X .

3

Exercise 1.1. Let X = {1, 2, 3}, how many members are there in P(X)?What if X = {1, . . . , n}?

Proposition 1.1 (De Morgan’s Rule). The complement of the union is theintersection of the complements, and the complement of the intersection isthe union of the completments, that is,

(A ∪B)c = Ac ∩Bc, and (A ∩B)c = Ac ∪Bc.

1.2 Mappings between sets

Given two set A and B, a correspondence that assigns to each member of Aa member of B is called by a mapping or function from A to B, denotedby f : A→ B. When B is the set of real numbers, we always use “function”.Given the function f : A→ B, for any x ∈ A, denote the member of B thatis assigned to x by f(x). For any subset X ⊆ A, define the image of Xunder f as

f(X) = {b ∈ B | ∃x ∈ X, f(x) = b}.

We call A the domain of f and f(A) the range of f . If f(A) = B, then thefunction f is said to be onto. If

∀a1, a2 ∈ A, a1 6= a2 ⇒ f(a1) 6= f(a2),

then the function f is said to be one-to-one. A mapping f : A → B thatis both one-to-one and onto is said to be invertible. Given an invertiblemapping f : A → B, ∀b ∈ B, there exists exactly one element a ∈ A suchthat f(a) = b, denoted by f−1(b). This assignment defines the mappingf−1 : B → A, called the inverse of f . If there is an invertible mappingfrom A to B, then we say A and B are equipotent, which,roughly speaking,means that they have the same number of elements.

Given two mappings f : A→ B and g : C → D for which f(A) ⊆ C thenthe composition g ◦ f : A→ D is defined by

(g ◦ f)(x) = g(f(x)),∀x ∈ A.

It is easy to see that the composition of invertible mapping is invertible.

4

1.3 Equivalence Relations*

Given two nonempty sets A and B, the Cartesian product of A with B isdefined by

A×B = {(a, b) | a ∈ A, b ∈ B}.

For example, R2 = {(x1, x2) | x1 ∈ R, x2 ∈ R} is the Cartesian productR × R. For a nonempty set X, we call a subset R ⊆ X ×X a relation onX and write xRx′ if (x, x′) ∈ R. The relation R is said to be reflecxive if

∀x ∈ X, xRx;

the relationship is said to be symmetric if

xRx′ ⇒ x′Rx;

the relation is said to be transitive if

xRx′, x′Rx′′ ⇒ xRx′′.

Exercise 1.2. In the 2-D plane, show the relation “ ≥ ” as a subset of R2.Is it reflecxive, symmetric, or transitive?

Definition 1.1. A relation R on a nonempty set X is called an equivalencerelation provided it is reflexive, symmetric, and transitive.

Definition 1.2. A relation R on a nonempty set X is called a partialordering provided it is reflexive, transitive, and

∀x, x′ ∈ X, xRx′, x′Rx⇒ x = x′.

Remark. Consider the rational preference�. It is reflexive, transitive, but notsymmetric; therefore it is not a equivalence relation. x � y, y � x ⇒ x ∼ ybut we may have x 6= y in general; therefore it is not a partial ordering.

1.4 The real numbers

Denote the set of real number by R. For any a, b ∈ R, the sum a + band the product ab are well-defined. We assume that R, with the sum andproduct, satisfies three types of axioms: Field Axioms, Positivity Axioms,and Completeness Axioms.

5

The Field Axioms.

Commutativity of Addition: ∀a, b ∈ R, a+ b = b+ a.

Associativity of Addition: ∀a, b, c ∈ R, (a+ b) + c = a+ (b+ c).

The Additive Identity: ∀a ∈ R, a+ 0 = a.

The Additive Adverse: ∀a ∈ R,∃b ∈ R, a+ b = 0. Denote b by −a.

Commutativity of Multiplication: ∀a, b ∈ R, ab = ba.

Associativity of Multiplication: ∀a, b, c ∈ R, (ab)c = a(bc).

The Multiplication Identity: ∀a ∈ R, 1a = a

The Multiplication Adverse: ∀a 6= 0,∃b ∈ R, ab = 1. Denote b by a−1.

The Distributive Property: ∀a, b, c ∈ R, a(b+ c) = ab+ ac).

The Nontriviality Assumption: 1 6= 0.

Remark. Any set with the operations sum and product that satisfies theseaxioms is called a field. You can check that with the conventional sum andproduct, the set of the rational number Q is a field, while the set of integersN is not.

The Positivity Axioms.

There is a set of real numbers, denoted by P , called the positive numbers,satisfying the following properties:

P1 If a and b are positive, then ab and a+ b are positive.

P2 For a real number a, exactly one of the following is true: a is positive,−a is positive, a = 0.

Define an ordering of the real number by a > b when a− b is positive and soon, and the intervals

(a, b) = {x | a < x < b}; [a, b) = {x | a ≤ x < b}(a, b] = {x | a < x ≤ b}; [a, b] = {x | a ≤ x ≤ b}

6

Remark. Also, the rational number Q satisfies the positivity axioms, whichmeans that we need other property to distinguish R from Q.

With the order defined aboved, we define the absolute value of a realnumber x by

|x| =

{x, x ≥ 0

−x, x < 0

Then we have the following inequality, called the Triangle Inequality:∀a, b ∈ R, |a+ b| ≤ |a|+ |b|.

The Completeness Axioms.

A nonempty set E ⊆ R is said to be bounded above if ∃b ∈ R,∀x ∈ E, x ≤b: the number b is called an upper bound for E. The smallest upper bound,if exists, is called the least upper bound, or the supremum of E, denotedby supE.

Exercise 1.3. Similarly, define the set E that is bounded below, the lowerbound and the greatest lower bound or infimum, inf E.

For any set that is bounded above, the axiom of completeness asserts thatits supremum exists.

The Completeness Axiom Let E ⊆ R be bounded above, its least upperbound, or supremum exists.

Example 1.1. Consider E = [0, 2). Then 2 is a upper bound. For any upperbound b, we have b ≥ 2: otherwise, we have b < b+2

2∈ E, contradicted.

Therefore, we 2 is the least upper bound, or supE = 2.

Remark. Consider the set E1 = {x ∈ Q | x2 < 2} and E2 = {x ∈ R | x2 < 2}.Both sets are bounded above. It can be show that in supE2 =

√2, but√

2 /∈ Q. No least upper bound for E1. In other words, Q does not satisfiedthe completeness axiom.

A taste of ε− δ language*

Let E ⊆ R be a nonempty and bounded above. Then if y = supE then∀ε > 0,∃x ∈ E such that

y − ε < x < y.

Exercise 1.4. Write the counterpart for infimum.

7

2 Basic Linear Algebra

Linear algebra studies the linear transformation on vector spaces, whichcan be represented by matrix. We will discuss what these terms mean.

2.1 Vector Spaces

Given a set V of objects, called vectors, suppose there are two operations:vector addition such that the sum of the vectors x and y is a vector denotedby x + y, and scalar multiplication such that the product of the scalar(i.e., real number) c and the vector x is a vector denoted by cx.

The set V , together with these two operations, is called a vector space(or linear space) if the following properties hold for all vectors x,y, z ∈ Vand all scalars c, d:

(1) x + y = y + x.

(2) x + (y + z) = (x + y) + z.

(3) There is a unique vector 0 such that 0 + x = x for all x.

(4) x + (−1)x = 0.

(5) 1x = x.

(6) c(dx) = (cd)x.

(7) (c+ d)x = cx + dy.

(8) c(x + y) = cx + cy.

One example of a vector space is the set Rn with component-wise addition andmultiplication by scalars. That is, if x = (x1, . . . , xn) and y = (y1, . . . , yn),then

x + y = (x1 + y1, . . . , xn + yn),

cx = (cx1, . . . , cxn).

If V is a vector space, then a subset W of V is called a linear subspace(or simply, a subspace) of V if for all x,y ∈ W and every scalar c, we have

x + y ∈ W, cx ∈ W.

8

In this case, W itself satisfies properties (1)-(8) if we use the operations thatW inherits from V , so that W is a vector space in its own right.

Exercise 2.1. Give two examples of subspaces of R2.

Let V be a vector space. A set a1, . . . , am of vectors in V is said to spanV , denoted by V = span{a1, . . . , am}, if ∀x ∈ V , there exists at least onem-tuple of scalars c1, . . . , cm such that

x = c1a1 + . . .+ cmam.

In this case, we say that x can be written as a linear combination of thevectors a1, . . . , am.

Example 2.1. Let V = R2, the set of vectors a1 = (1, 0), a2 = (1, 1) spanV .

The set a1, . . . , am of vectors is said to be independent if to each x ∈ Vthere corresponds at most one m−tuple of scalars c1, . . . , cm such that

x = c1a1 + . . .+ cmam.

Proposition 2.1. The set {a1, . . . , am} is independent provided

0 = d1a1 + . . .+ dmam =⇒ d1 = d2 = . . . = dm = 0.

If the st of vectors a1, . . . , am both span V and is independent, it is saidto be a basis for V .

Theorem 2.2. Suppose V has a basis consisting of m vectors. Then any setof vectors that span V has at least m vectors, and any set of vectors that isindependent has at most m vectors. In particular, any basis for V has exactlym vectors.

If V has a basis consisting of m vectors, we say m is the dimension of V ,denoted by dimV . We make the convention that the vector space consistingof the zero vector alone has dimension zero. It is easy to see that Rn hasdimension n. The following set of vectors is called the standard basis for Rn:

e1 = (1, 0, 0, . . . , 0),

e2 = (0, 1, 0, . . . , 0),

. . .

en = (0, 0, 0, . . . , 1).

9

The vector space Rn has many other bases, but any basis of Rn must consistof precisely n vectors.

Because Rn has a finite basis, so does every subspace of Rn. This fact isa consequence of the following theorem:

Theorem 2.3. Let V be a vector space of dimension m. If W is a lin-ear subspace of V (different from V ), then W has dimension less than m.Furthermore, any basis a1, . . . ,ak for W may be extended to a basis

a1, . . . , ak,ak+1, . . . ,am

for V .

2.2 Inner Products

If V is a vector space, an inner product on V is a function assigning, toeach pair of x,y of vectors of V ,a real number denoted by 〈x,y〉, such thatthe following properties hold for all x,y, z ∈ V and all scalars c:

(1) 〈x,y〉 = 〈y,x〉.

(2) 〈x + y, z〉 = 〈x, z〉+ 〈y, z〉.

(3) 〈cx,y〉 = c〈x,y〉 = 〈x, cy〉.

(4) 〈x,x〉 > 0 if x 6= 0.

A vector space V together with an inner product on V is called an innerproduct space.

A given vector space may have many different inner products. Oneparticularly useful inner product on Rn is defined as follows: for all x =(x1, . . . , xn), y = (y1, . . . , yn), define

〈x,y〉 = x1y1 + . . .+ xnyn.

This is the inner product we commonly use. It is also called the dot productand denoted by x · y or x′y.

Exercise 2.2. Verify the the inner product defined above satisfies the prop-erties (1)-(4).

10

If V is an inner product space, one can defines the length (or norm) ofa vector of V by

‖x‖ = 〈x,x〉12 .

You can verify that the norm function has the following properties:

(1) ‖x‖ > 0 if x 6= 0.

(2) ‖cx‖ = |c|‖x‖.

(3) ‖x + y‖ ≤ ‖x‖+ ‖y‖.

The last inequality is called the triangle inequality, which can be provedby the Cauchy-Schwarz inequality

〈x,y〉 ≤ ‖x‖‖y‖.

The proof of Cauchy-Schwarz Inequality. For any a ∈ R, we have 〈ax +y, ax + y〉 ≥ 0. That is

‖x‖2a2 + 2〈x,y〉a+ ‖y‖2 ≥ 0,∀a ∈ R

Then we have

∆ = 4〈x,y〉2 − 4‖x‖2‖y‖2 ≤ 0⇒ 〈x,y〉 ≤ ‖x‖‖y‖

The proof of Triangle Inequality.

‖x + y‖2 = ‖x‖2 + 2〈x,y〉+ ‖y‖2

≤ ‖x‖2 + 2‖x‖‖y‖+ ‖y‖2

= (‖x + y‖)2

Remark. Any function from V to real number that satisfies properties (1)-(3) just listed is called a norm on V . The length function derived from aninner product is one example of a norm, but there are other norms that arenot derived from inner products. On Rn, for example, one has not only thefamiliar norm derived from the dot product, which is called the euclideannorm, but one has also the sup norm, defined by

|x| = max{|x1|, . . . , |xn|}.

11

2.3 Matrices

A matrix is a rectangular array of numbers.

A =

a11 . . . a1n...

. . ....

am1 . . . amn

is called an m×n matrix, sometimes denoted by (aij)m×n. Let B = (bij)m×n,define the sum of A and B by a11 + b11 . . . a1n + b1n

.... . .

...am1 + bm1 . . . amn + bmn

and the product of A and real number c by

cA =

ca11 . . . ca1n...

. . ....

cam1 . . . camn

With these operation, the set of all m×n matrices is a vector space, denotedby M(m,n).

The set of matrices has an additional operation called matrix multipli-cation. For Am×n and Bn×p, the product AB is defined to be the matrixC = (cij)m×p where

cij =n∑k=1

aikbkj.

Define the identical matrix Ik = (δij)k×k where

δij =

{1 i = j

0 i 6= j

Then it has the form

Ik =

1 0 . . . 00 1 . . . 0...

.... . .

...0 0 . . . 1

You can verify that the matrix multiplication satisfies the following proper-ties:

12

(1) A(BC) = (AB)C

(2) A(B + C) = AB + AC

(3) (A+B)C = AC +BC

(4) (cA)B = c(AB) = A(cB)

(5) For any Am×n,ImA = A, and AIn = A.

Given a matrix A = (aij)m×n, define its transpose by D = (dij)n×m withdij = aji,denoted by Atr (or A′ or At). Then we have

(1) (Atr)tr = A.

(2) (A+B)tr = Atr +Btr.

(3) (AC)tr = CtrAtr.

2.4 Linear Transformations

Let V and W be two vector spaces, a mapping T : V → W is called a lineartransformation if it satisfies: ∀x,y ∈ V, ∀c ∈ R:

(1) T (x + y) = T (x) + T (y)

(2) T (cx) = cT (x).

The following theorem asserts that a linear transformation is uniquelydetermined by the values on basis elements.

Theorem 2.4. Let V be a vector spaces with basis a1, . . . , am. Let W bea vector space. Given any m vectors b1, . . . ,bm, there is exactly one lineartransformation T : V → W such that

T (ai) = bi,∀i = 1, . . . ,m

Proof. First we show the existence by constructing a function T : V → Win the following way. Let

T (ai) = bi,∀i = 1, . . . ,m

13

and for any x =∑n

i=1 ciai ∈ V , let

T (x) =n∑i

ciT (ai) =n∑i

cibi.

Then T : V → W is a linear transformation.Next we show that the linear transformation is unique. Suppose there

are two linear transformation T : V → W and S : V → W such that

T (ai) = S(ai) = bi,∀i = 1, . . . ,m

the for any x =∑n

i=1 ciai ∈ V ,

T (x) =n∑i

ciT (ai) =n∑i

cibi =n∑i

ciS(bi) = S(x).

The matrix representation of a linear transformation.

Any linear transformation from one finite dimensional mapping to anothercan be represented by matrices.

Let’s first look at the special case. Let A ∈ M(m,n), the function T :Rn → Rm defined by

T (x) = Ax, ∀x ∈ Rn

is a linear transformation. You can check it using the properties of matrixmultiplications.

On the other hand, for any linear transformation T : Rn → Rm, thereexists some matrix A ∈M(m,n) such that T (x) = Ax, ∀x ∈ Rn.

Proof. Let bi = T (ei),∀i = 1, . . . , n where e1, . . . , en is the standard basis forRn, and construct a matrix A = [b1 . . .bn] ∈ M(m,n). Then the mappingS(x) = Ax is a linear transformation from Rn to Rm with

S(ei) = bi,∀i = 1, . . . , n.

By Theorem 2.4, we have T (x) = S(x) = Ax.

14

In general, let T : V → W be linear, v1, . . . ,vm be a basis for the vectorspace V , and w1, . . . ,wn be a basis for W . For any x ∈ V , there are uniquenumbers x1, . . . , xm such that

x = x1v1 + . . .+ xmvm.

Let y = T (x) ∈ W , then y can be expressed as a unique linear combination

y = y1w1 + . . .+ ynwn.

Also, T (vj) ∈ W , there are unique numbers a1j, . . . , anj such that

T (vj) = a1jw1 + . . .+ anjwn.

Then

n∑i=1

yiwi = T (x) = T (m∑j=1

xjvj) =m∑j=1

xjT (vj)

=m∑j=1

xj

n∑i=1

aijwi =n∑i=1

(m∑j=1

aijxj)wi

Therefore, yi =∑m

j=1 aijxj,∀i. Denote x = (x1, . . . , xm), y = (y1, . . . , yn),A = (aij) ∈M(n,m). Then

y = T (x) = Ax.

Therefore, the matrix A represents the linear transformation T : V → W .Furthermore, it can be seen that the matrix A depends on the bases we usedfor V and W . In general, the representing matrix would be different if wechoose a different basis for V or W .

Example 2.2. The identity mapping id : V → V defined by

id(x) = x,∀x ∈ V

is a linear transformation. It can be represented by the identity matrix Ikwhere k is the dimension of V , no matter which basis is used.

15

Composition and inverse of linear transformation.

Let V,W,U be three vector spaces with dimension m,n and p, and T : V →W and S : W → U be two linear transformation, you can show that thecomposite S ◦ T : V → U is also linear.

Given the bases {v1, . . . ,vm}, {w1, . . . ,wn}, and {u1, . . . ,up}, supposeT and S are represented by matrices An×m and Bp×n. Then the compositeS ◦ T is represented by the matrix multiplication BA = (cij)p×m.

Proof. We have S ◦ T : V → U is linear. Suppose that it is represented bythe p×m matrix C. We want to show that C = BA. Since

T (vj) = a1jw1 + . . .+ anjwn,

S(wk) = b1ku1 + . . .+ bpkup.

we have

p∑i=1

cijui = S(T (vj)) = S(n∑k=1

akjwk) =n∑k=1

akjS(wk)

=n∑k=1

akj

p∑i=1

bikui =

p∑i=1

(n∑k=1

bikakj)ui

Therefore, cij =∑n

k=1 bikakj,∀i, j. Then C = BA.

For any invertible linear transformation T : V → W , its inverse mappingT−1 : W → V is also linear. Also, in the following exercises you may showthat if {v1, . . . ,vn} is basis for V , then {T (v1), . . . , T (vn)} is basis for Wprovided that T is invertible.

Exercise 2.3. Show that T−1 : W → V is linear.

Exercise 2.4. Suppose {v1, . . . ,vn} is basis for V , T : V → W is invertible.Show that {T (v1), . . . , T (vn)} span the vector space W .

Exercise 2.5. Suppose {v1, . . . ,vn} is basis for V , T : V → W is invertible.Show that {T (v1), . . . , T (vn)} is independent.

16

Then, W must have the same dimension as V . Therefore, T can berepresented by a n×n matrix, denoted byAn×n. Suppose T−1 is representedby Bn×n. By the definition of inverse mapping, we have

T ◦ T−1 = id : W → W

T−1 ◦ T = id : V → V

Then by the previous result for composite mapping and the identity mappingon Example 2.2, we have

AB = In = BA.

For any matrix An×n, if there exist another matrix Bn×n such that AB =BA = In, then the matrixA is said to be invertible, and B is the inversematrix of A, denoted by A−1. With this definition, we have T−1 is repre-sented by A−1 provided that T : V → V , represented by A, is invertible.

Furthermore, the invertible linear transformation is also called a linearisomorphism, and two vector spaces are isomorphic if there exists anlinear isomorphism between them.

For any vector space V with basis v1, . . . ,vm, there exists an invertiblemapping T : V → Rm such that T (vi) = ei, i = 1, . . . ,m. Therefore, anyvector spaces with dimension m are isomorphic to Rm, which means thatthey the same structure as the Euclidean space with the same dimension.

2.5 The Four Fundamental Subspaces.

Let T : V → W is linear. The range of T is defined by

{T (x) = Ax ∈ W | ∀x ∈ V } ⊆ V

as we discussed in Section 1.2. Also, we can define the kernel of T , denotedby kerT , as the set of vectors that has 0W as image. That is

kerT = {x ∈ V | T (x) = 0W} ⊆ V.

It is easy to check that the range of T is a subspace of W , and that the kernelof T is a subspace of V , which is also called the null space of T .

Exercise 2.6. Show that 0 ∈ kerT , and that kerT is a subspace of V .

Define the rank of the T by the dimension of the range of T , and thenullity of T by the dimension of the null space of T . Then we have thefollowing result called the rank-nullity theorem.

17

Theorem 2.5 (Rank-Nullity Theorem). Let T : V → W be a linear trans-formation, then

rankT + nullity T = dimV

Sketch of the proof. Suppose dimV = n, and nullity T = k, we want to showthat rankT = n−k. Let v1, . . . ,vk be a basis for the null space. Then it canbe extended to a basis for V , v1, . . . ,vk,vk+1, . . . ,vn. Then it is sufficient toshow that T (vk+1), . . . , T (vn) is a basis for the range of T .

For any matrix Am×n, rewrite

A = [a1 . . . an] =

a1...

am

where ai is the ith row of A and aj is the jth column of A. Define the columnspace by span{a1, . . . , an}, and call its dimension the column rank of A.Then, it can be seen that the column space of the matrix A is the same asthe range of the linear transformation T by

T (x) = Ax = [a1 . . . am]

x1. . .xm

= x1a1 + . . .+ xmam.

Therefore, the column rank of a matrix A is equal to the rank of the lineartransformation T (x) = Ax.

Define the row space as span{a1, . . . , am} and call its dimension the rowrank of A. The row space of A is the column space of Atr, the range of themapping S(y) = Atry : Rm → Rn.

One fundamental result is that for any matrix the row rank is equal tothe column rank, which is called the rank of the matrix. Therefore, it isthe same as the rank of the corresponding linear transformation. It is clearthat the row rank of Am×n is less than or equal to m and the column rank isless than or equal to n. Therefore, we have

rankA ≤ min{m,n}.

If the equality holds, the matrix is said to be full rank.For a square matrix An×n, it is full rank when rankA = n and nullityA =

0. Therefore, it is invertible (check it).

Remark. For any matrix Am×n, the null spaces and the column spaces of Aand Atr are called the four fundamental subspaces.

18

Linear equation system and linear transformation.

The system of linear equations with n variables and m equations can beexpressed in matrix form,

Ax = b (1)

where A ∈M(m,n). Let T (x) = Ax. Consider the null space of T . We haveany member in the null space is a solution to the homogeneous system

Ax = 0 (2)

Since 0 is always in the null space, the equation (2) at least has one solutionx = 0. If the nullity of T is positive, then there are infinite number ofsolutions to (2). If the nullity of T is 0, then x = 0 is the unique solution to(2). Next, consider the range of T , or the column space of A. If b is not inthe column space, then there is no solution to (1). If b belongs to it, thenthe equation (1) has at least one solution, say xb. Then all the vectors in thenull space plus xb also solve the equation (1).

IfAn×n is a square and full rank matrix, that is rankA = n and nullityA =0, and A is invertible, therefore, the (1) has an unique solution for any b,

xb = A−1b.

2.6 Determinant and Matrix Inversion.

The determinant is a function that assigns, to each square matrix A, a num-ber called the determinant of A and denoted detA or |A|, if it satisfies thefollowing properties:

(1) detA is multilinear in the column vectors of matrix A. That is,

det[. . . ,bi + ci, . . . , ] = det[. . . ,bi, . . .] + det[. . . , ci, . . .]

det[. . . , λai, . . .] = λ det[. . . , ai, . . .]∀λ ∈ R

(2) Alternating :

det[. . . , ai, ai+1, . . .] = − det[. . . , ai+1, ai, . . .]

(3) For any identity matrix In, det In = 1.

19

It can be shown that there exists exactly one function satisfies all theseproperties. Therefore, the determinant of any square matrix is well-defined.There are different ways to define this function. Here we will use inductionand the so called Laplace expansion formula.

• For n = 1, A = [a], detA = a.

• For n = 2, A =

[a11 a12a21 a22

], detA = a11a22 − a12a21.

• For n ≥ 3, define the (i, j)−cofactor of A by

Cij = (−1)i+j detAij

where Aij is the the determinant of the (n − 1) × (n − 1) matrix thatresults from deleting the i-th row and the j-th column of A, called the(i, j)−minor of A . Then

detA = ai1Ci1 + . . .+ ainCin.

You can check that the function defined above satisfies all the three propertiesof the determinant.

Example 2.3. For n = 3

det

a11 a12 a13a21 a22 a23a31 a32 a33

= a11 det

[a22 a23a32 a33

]−a12 det

[a21 a23a31 a33

]+a13 det

[a21 a22a31 a32

].

We have the following properties of the determinant:

(1) detA = detAtr.

(2) detAB = detA detB.

(3) det[. . . , ai, . . . , aj . . .] = 0, when ai = aj, i 6= j.

(4) detA 6= 0 if and only if A is invertible.

Remark. The geometric interpretation of the determinant. Writing A =[a1, . . . , an] ∈M(n, n), we have the absolute value of detA is the n-dimensionalvolume of the parallelepiped spanned by the vectors a1, . . . , an.

20

Cramer’s rule and a formula for A−1

Let An×n = [a1, . . . , an], then the equation system Ax = b can be rewrittenas

x1a1 + . . .+ xna

n = b.

Then for each i we have

xi det[a1, . . . , an] = det[a1, . . . ,b, . . . , an]

where b occurs in the ith place. This is the so-called Cramer’s rule, whichcan be used to solve the system of linear equations when detA 6= 0.

Proof.

det[a1, . . . ,b, . . . , an] = det[a1, . . . ,∑

xkak, . . . , an]

=∑

det[a1, . . . , xkak, . . . , an]

=∑

xk det[a1, . . . , ak, . . . , an]

= xi det[a1, . . . , ai, . . . , an]

Now assume that detA 6= 0 so there exists B = (bij) ∈M(n, n) such thatAB = In. Write ej ∈ Rn for the column vector with 1 in the jth place andzeros elsewhere. We have

bij =det[a1, . . . , ej, . . . , a

n]

detAwhere the ej occurs in the ith position. This can be shown by

AB = I ⇒ Abj = ej,∀jand the Cramer’s rule. By expanding det[a1, . . . , ej, . . . , a

n] down the ithcolumn using the Laplace expansion, we can show that

bij =(−1)i+j det(Aji)

detA

where the Aji is (j, i) minor of A, obtained from A by deleting the jth rowand ith column.

Example 2.4. For example, let A =

[a bc d

]and assume detA 6= 0, then we

have

A−1 =1

ad− bc

[d −b−c a

].

21

2.7 Eigenvectors and Eigenvalues*

Let V be a vector space and T : V → V be linear transformation representedby the square matrix A with respect to some basis. A vector v ∈ V is calledan eigenvector of T is v 6= 0 and T (v) = λv for some λ ∈ R. We call λ ∈ Ran eigenvalue of T if T (v) = λv for some nonzero vector v ∈ V .

Using the terminologies we defined in section 2.5, we have the followingstatements are equivalent.

(1) λ is an eigenvalue of T ;

(2) The null space of T − λI is not {0};

(3) The linear transformation T − λI is invertible;

(4) det(T − λI) = 0.

Then we define the characteristic polynomial by det(A−xIn). We denotethe characteristic polynomial of the linear transformation T by χT (x), andof a matrix A by χA(x). Then we have λ is an eigenvalue of T if and only ifλ is a root of χT (x), that is χT (λ) = 0.

Example: An Application on Marxian Economics

Consider a input-output matrix A = (aij) ∈M(n, n) where aij is the amountof good i required to produce 1 unit of good j, and the labor vector l =(l1, . . . , ln) where lj is the amount of labor required to produce 1 unit of goodj. Define the value of good j, λj as the total amount of labor directly andindirectly required to produce one unit of good j, then we have

Λ = ΛA+ l

where Λ = (λ1, . . . , λn) is the value vector. Therefore, we have

Λ = (I − A)−1l

if (I − A)−1 exists.Let p be the row vector of prices, and b the column vector of real wage,

then pA + pbl is the row vector of unit cost. Assume the uniform rate ofprofit is π, then we have

p = (1 + π)(pA+ pbl) = (1 + π)p(A+ bl)

22

Therefore, the price p is a eigenvector of the matrix M ≡ A + bl, and therate of profit is determined by the corresponding eigenvalue. To make thesevariables meaningful, all the prices should be nonnegative. This is guar-anteed by the so-called Perron-Frobenius Theorem when the matrix Mis nonnegative. Some famous results in Marxian economics like the Funda-mental Marxian Theorem and the Okishio’s Theorem can be derived in thisframework.

23

3 Rudiments of Analysis

In this section, we will review the basic topics of analysis, including theconvergence of sequence, the continuity and differentiability of the real-valuedfunction.

3.1 Convergence of Sequences

A sequence of real number is a function x : N+ → R that assigns a realnumber x(n) to each n = 1, 2, . . ., denoted by (xn) = (x1, x2, . . . , ).

Example 3.1. Let xn = sinn2n+1

. Then the sequence looks like

(1

3sin 1,

1

5sin 2,

1

7sin 3 . . . , )

Given the sequence (xn) and (yn), we have (xn + yn), (−xn), (xnyn) and,provided every yn is non-zero, (xn/yn). Also, we may have (cxn) for anyconstant c, and (|xn|).

We want to analyze how the terms xn of the sequence behavior as n getarbitrarily large and specifically whether or not the terms approach arbitrarilyclosely some limiting value.

First, we don’t care what the values of the first few terms are, or whatthe first 100 million terms are. That is, the long-run behavior of the termsof a sequence is not is not affected by chopping the first k terms. For anysequence (xn) and any k ∈ N, let yn = xn+k for all n. We call (yn) a tail of(xn).

Next, we can capture “arbitrarily close to” via ε. From the definition ofabsolute value of real number in Section , we have, for any x, L ∈ R andε > 0,

|x− L| < ε⇐⇒ L− ε < x < L+ ε

That is x lies within a distance ε of L. To formulate that x is arbitrarilyclose to L, we allow ε > 0 to be very small.

Now we have a precise, formal definition of convergence. Let (xn) be asequence of real numbers and let L ∈ R. Then we say (xn) converges toL, denoted by limn→∞ xn = L or xn → L, n→∞, if

∀ε > 0 ∃N ∈ N ∀n > N |xn − L| < ε.

24

When xn → L, n→∞, we say L is the limit of (xn).For any (xn), if there exists L ∈ R such that xn → L as n → ∞, we

say (xn) converges; otherwise, we say (xn) diverges. For convenience, wedefine the two specific cases of divergence. We say xn tends to infinity andwrite xn →∞ as n→∞ if

∀M ∈ R,∃N ∈ N,∀n ≥ N, xn > M

Similarly we can define xn → −∞ if

∀M ∈ R, ∃n ∈ N,∀n > N, xn < M

If a sequence of real numbers converges, we can show that the limit isunique. The proof of this statement is a good example to show the construc-tion of ε−N proofs.

Example 3.2. Let (xn) be a sequence and suppose that xn → L1 and xn →L2 as n→∞. Then L1 = L2.

Proof. Suppose L1 6= L2. Take ε = 12|L1 − L2|. Since xn → L1,

∃N1 ∈ N,∀n > N1, |xn − L1| < ε.

Similarly, since xn → L2, we have

∃N2 ∈ N,∀n > N2, |xn − L2| < ε.

Let N = max{N1, N2}, then we have ∀n > N ,

|L1 − L2| = |L1 − xn + xn − L2| ≤ |L1 − xn|+ |xn − L2| < ε = |L1 − L2|

contradicted.

In the following, we have some more exercises to help you master theconstruction of ε−N proofs.

Exercise 3.1. Let (xn) be a sequence and suppose that xn → L as n→∞,show that |xn| → |L|, as n→∞.

Hints. Show that ||a|− |b|| ≤ |a−b| by reversing the Triangle Inequality.

Also, we can show the preservation of weak inequalities : assume thatxn → a, yn → b and xn ≤ yn,∀n, then a ≤ b.

25

Proof. Suppose, on the contrary, a > b and let ε = 12(b − a). Then we have

N1 and N2 such that

∀n > N1, |xn − a| < ε⇒ xn > a− ε∀n > N2, |yn − b| < ε⇒ yn < b+ ε

Therefore, take N = max{N1, N2}, for n > N , we have

a− ε < xn ≤ yn < b+ ε

then a− b < 2ε = a− b contradicted.

However, it should be noted that the limit does not preserve the strictinequalities. This can be seen from the following example.

Example 3.3. Let xn = 0, yn = 1n, then we have xn < yn, but xn → 0, yn →

0.

Exercise 3.2 (The Sandwiching Rule.). Let (xn), (yn), (zn) be three se-quences of real numbers. Suppose xn → L, yn → L, and xn ≤ zn ≤ ynfor all n, then show that zn → L, as n→∞.

As we can see from the definition of convergence, a sequence that isconvergent will get “trapped” into the neighbor of limit after chopping thefirst finite number of elements. Therefore, it should be bounded in the sensethat

∃M ∈ R,∀n, |xn| ≤M.

Proof. Assume that xn → L and take ε = 1. Then there exists N such that

∀n > N, |xn − L| < 1 =⇒ |xn| ≤ |L|+ 1

Let M = max{x1, . . . , xN , |L|+ 1, then we have |xn| ≤M, ∀n.

Since any convergent sequence is bounded, then if a sequence is notbounded, it must diverge. However, the sequence that diverges can bebounded, e.g., yn = (−1)n. In this case, we usually show the divergenceof a sequence by constructing subsequences.

A subsequence (yk) of the sequence (xn) is defined by a strictly increas-ing mapping f : N→ N, so that

yk = xnk,

26

where nk = f(k). The strictly increasing mapping f is the rule of choiceby which terms of xn are selected to form the subsequence. It is strictlyincreasing to maintain the same order and make sure that each term of xnis selected once. Formally, the subsequence yk is the composite of this thestrictly increasing mapping f and the function x(n), that is y = x◦f : N→ Rwith y(k) = x(f(k)).

Every subsequence of a convergent sequence is convergent. Furthermore,it converges to the same limit of the the sequence. The contrapositive of thisstatement is usually used to establish the divergence. That is, if a sequencesubsequences which converge to different limits, then the sequence does notconverge.

Example 3.4. Let

xn = (−1)nn2

n2 + 1

Then x2n → 1 and x2n+1 → −1. Therefore, xn diverges.

The algebra of limits (AOL)

Suppose xn → a, yn → b as n→∞ and c is a constant number, then we have

(1) If xn = c, then a = c.

(2) xn + yn → a+ b.

(3) cxn → ca.

(4) xn − yn → a− b.

(5) xnyn → ab.

(6) If b 6= 0 and yn 6= 0, then 1/yn → 1/b.

(7) If b 6= 0 and yn 6= 0, then xn/yn → a/b.

Here we only provide the proof of (5), and the rest are left as exercises forthe reader.

27

Proof of (5). Since yn convergent, it is bounded. So we can find M ∈ R suchthat |yn| < M . For any ε > 0, from xn → a and yn → b, we can find N1 andN1 such that

∀n ≥ N1, |xn − a| <ε

2M

∀n ≥ N2, |yn − b| <ε

2|a|

Therefore, for all n > max{N1, N2}

|xnyn − ab| = |xnyn − ayn + ayn − ab|≤ |xn − a||yn|+ |a||yn − bn|≤M |xn − a|+ |a||yn − bn|≤ ε

A typical application of AOL is given by the following example.

Example 3.5. Let xn = n2+n+13n2+4

. Then

xn =n2(1 + 1

n+ 1

n2 )

3n2(1 + 43n2 )

=1

3(1 + 1

n+ 1

n2

1 + 43n2

)→ 1

3(1 + 0 + 0

1 + 0) =

1

3

Monotonic Sequences

Let (xn) be a sequence of real number, then it is said to be monotonicincreasing if xn ≤ xn+1,∀n, and monotonic decreasing if xn ≥ xn+1,∀n.It is monotonic if it is either monotonic increasing or monotonic decreasing.Then we have the following theorem asserts that any monotonic sequenceconverges if and only if it is bounded.

Theorem 3.1 (Monotonic Sequence Theorem). Let (xn) be a sequence ofreal numbers.

(1) Assume (xn) is monotonic increasing. Then (xn) converges if and onlyif it is bounded above, i.e., ∃M,∀n, |xn| ≤M .

28

(2) Assume (xn) is monotonic decreasing. Then (xn) converges if and onlyif it is bounded below.

We prove the first part, recalling the completeness axiom for real numberin Section 1.4 and the condition for supremum in terms of ε− δ language.

Proof. Let E = {xn | n ∈ N}. Then we have E is a nonempty subset ofR and bounded above since (xn) is bounded above. By the completenessaxiom, supS exists. Let L = supS, then ∀ε > 0, ∃xN ∈ E such that

L− ε < xN ≤ L.

Therefore, ∀n > N ,

L− ε < xN ≤ xn ≤ L =⇒ |xn − L| < ε,

since xn is monotonic increasing. Then we have xn → L.For the converse we use the fact that any convergent sequence is bounded.

The Bolzano-Weierstrass Theorem

Since monotonic real subsequences behave so well, the following fact is wel-come: any real sequence has a monotonic subsequence. This is the so-calledscenic viewpoint theorem. Once it is established, we have the famous re-sult called the Bolzano-Weierstrass Theorem asserting that any boundedreal sequence has convergent subsequence, by applying the monotonic se-quence theorem.

Now we first prove the scenic viewpoint theorem. Let (xn) be a realsequence. Then (xn) has a monotonic subsequence. We consider the set of“scenic viewpoint” or “peaks”, V = {k ∈ N | m > k ⇒ xm < ak}. Thatis, given any k ∈ V , looking toward infinity from a point at height xk, nohigher point would impede our view. Then V can be infinite or finite. Inboth cases, we can construct a monotonic subsequence of (xn) as follows.

Case 1: V is infinite. Then enumerate the elements of V as n1 < n2 < . . ..Then xnk

is a subseqence of (xn) and

r > s =⇒ nr > ns =⇒ xnr < xns

that is, (xnk) is monotonic decreasing.

29

Case 2: V is finite. Then ∃N ∈ N such that ∀k ∈ V, k < N . Take m1 = N ,then m1 /∈ V . Therefore, ∃m2 > m1 with xm2 ≥ xm1 since m1 isnot a “peak”. Since m2 /∈ V , there exists m3 > m2 such that xm3 ≥xm2 . Proceeding in this way, we can generate a monotonic increasingsubsequence (xmk

).

Let (xn) be a bounded real sequence. By the scenic viewpoint theorem, weknow that (xn) has a monotonic subsequence which is also bounded. Thenthis subsequence converges, by the Monotonic Sequence Theorem 3.1. Thenwe have

Theorem 3.2 (Bolzano-Weierstrass Theorem). Let (xn) be a bounded realsequence. Then (xn) has a convergent subseqence.

Cauchy Sequences.

The idea behind the Cauchy sequence is that if the terms of a sequenceare ultimately arbitrarily close to one another, then there should be a valueto which they converge. We say a sequence of real number is a Cauchysequence if

∀ε > 0,∃N ∈ N,∀m,n ≥ N, |xm − xn| < ε

This condition is also called the Cauchy condition. About Cauchy se-quences, we have the following facts.

(1) A Cauchy sequence is bounded.

(2) A convergent sequence is a Cauchy sequence.

(3) If a Cauchy sequence has a convergent subsequence, the Cauchy se-quence itself must also converge the limit of its subsequence.

Proof. The proof is left as an exercise, with solution provided below.

(1) Let (xn) be a Cauchy sequence. Take ε = 1, then existsN such that∀m,n ≥ N , we have |xm− xn| < 1. Thus we get |xn| ≤ |xN |+ 1 for alln ≥ N . Then we have |xn| ≤M where

M = max{|a1|, . . . , |aN−1|, |aN |+ 1}.

30

(2) Let xn → L. Then ∀ε > 0, ∃N ∈ N such that ∀k ≥ N , we have|xk − L| < ε

2. Then for any m,n ≥ N , we have

|xm − xn| = |xm − L+ L− xn| ≤ |xm − L|+ |xn − L| <ε

2+ε

2= ε.

(3) Let (xn) be a Cauchy sequence, and xnk→ L. For any ε > 0, ∃N1 such

that∀m,n ≥ N1, |xm − xn| <

ε

2.

Since xnk→ L, ∃N2 such that

∀k ≥ N2, |xnk− L| < ε

2.

Then fix k such that k ≥ max{N1, N2},then nk ≥ k ≥ N1. Therefore,

∀n > N, |xn−L| = |xn−xnk+xnk

−L| < |xn−xnk|+ |xnk

−L| < ε

2+ε

2.

The second fact show that any convergent sequence is a Cauchy sequence.The converse is true. Since any Cauchy sequence is bounded by the first fact,then it has convergent subsequence by Theorem 3.2. Together with the lastfact, we have it converges. Therefore, we have the Cauchy ConvergenceCriterion.

Theorem 3.3 (Cauchy Convergence Criterion). Let (xn) be a sequence ofreal number, then (xn) is convergent if and only if (xn) is a Cauchy sequence.

31