Appendix Matrix Algebra Review

Eric Zivot

Department of Economics

University of Washington

January 3, 2000

This version: February 6, 2001

1 Matrix Algebra Review

A matrix is just an array of numbers. The dimension of a matrix is determined by

the number of its rows and columns. For example, a matrix A with n rows and m

columns is illustrated below

ï£® ï£¹

a11 a12 . . . a1m

ï£¯ a21 a22 . . . a2m ï£º

ï£¯ ï£º

A =ï£¯ . .ï£º

.

ï£°. . ... . ï£»

. . .

(nÃ—m)

an1 an2 . . . anm

where aij denotes the ith row and j th column element of A.

A vector is simply a matrix with 1 column. For example,

ï£® ï£¹

x1

ï£¯ x2 ï£º

ï£¯ ï£º

x =ï£¯ . ï£º

ï£°.ï£».

(nÃ—1)

xn

is an n Ã— 1 vector with elements x1 , x2 , . . . , xn . Vectors and matrices are often written

in bold type (or underlined) to distinguish them from scalars (single elements of

vectors or matrices).

The transpose of an n Ã— m matrix A is a new matrix with the rows and columns

of A interchanged and is denoted A0 or A| . For example,

ï£® ï£¹

Â· Â¸ 14

123

, A0 = ï£° 2 5 ï£»

A=

456 (3Ã—2)

(2Ã—3)

36

1

ï£® ï£¹

1 Â£ Â¤

= ï£° 2 ï£», x0 =

x 123.

(3Ã—1) (1Ã—3)

3

A symmetric matrix A is such that A = A0 . Obviously this can only occur if A

is a square matrix; i.e., the number of rows of A is equal to the number of columns.

For example, consider the 2 Ã— 2 matrix

Â· Â¸

12

A= .

21

Clearly, Â· Â¸

12

A0 = A = .

21

1.1 Basic Matrix Operations

1.1.1 Addition and subtraction

Matrix addition and subtraction are element by element operations and only apply

to matrices of the same dimension. For example, let

Â· Â¸ Â· Â¸

49 20

A= , B= .

21 07

Then

Â· Â¸ Â· Â¸ Â· Â¸ Â· Â¸

49 20 4+2 9+0 69

A+B = + = = ,

21 07 2+0 1+7 28

Â· Â¸ Â· Â¸ Â· Â¸ Â· Â¸

4âˆ’2 9âˆ’0

49 20 29

Aâˆ’B = âˆ’ = = .

2âˆ’0 1âˆ’7 2 âˆ’6

21 07

1.1.2 Scalar Multiplication

Here we refer to the multiplication of a matrix by a scalar number. This is also an

element-by-element operation. For example, let c = 2 and

Â· Â¸

3 âˆ’1

A= .

05

Then Â· Â¸ Â· Â¸

2 Â· 3 2 Â· (âˆ’1) 6 âˆ’2

cÂ·A= = .

2 Â· (0) 2Â·5 0 10

2

1.1.3 Matrix Multiplication

Matrix multiplication only applies to conformable matrices. A and B are conformable

matrices of the number of columns in A is equal to the number of rows in B. For

example, if A is mÃ— n and B is m Ã— p then A and B are conformable. The mechanics

of matrix multiplication is best explained by example. Let

Â· Â¸ Â· Â¸

12 121

and B =

A= .

34 342

(2Ã—2) (2Ã—3)

Then

Â· Â¸Â· Â¸

12 121

AÂ·B Â·

=

34 342

(2Ã—2) (2Ã—3)

Â· Â¸

1Â·1+2Â·3 1Â·2+2Â·4 1Â·1+2Â·2

=

3Â·1+4Â·3 3Â·2+4Â·4 3Â·1+4Â·2

Â· Â¸

7 10 5

= =C

15 22 11 (2Ã—3)

The resulting matrix C has 2 rows and 3 columns. In general, if A is n Ã— m and B

is m Ã— p then C = A Â· B is n Ã— p.

As another example, let

Â· Â¸ Â·Â¸

12 2

A = 3 4 and B = 6 .

(2Ã—2) (2Ã—1)

Then

Â· Â¸Â· Â¸

12 5

AÂ·B Â·

=

34 6

(2Ã—2) (2Ã—1)

Â· Â¸

1Â·5+2Â·6

=

3Â·5+4Â·6

Â· Â¸

17

= .

39

As a & example, let

nal

ï£® ï£¹ ï£® ï£¹

1 4

x = ï£° 2 ï£», y = ï£° 5 ï£».

3 6

Then ï£®

ï£¹

4

Â£ Â¤

x0 y = 1 2 3 Â· ï£° 5 ï£» = 1 Â· 4 + 2 Â· 5 + 3 Â· 6 = 32

6

3

1.2 The Identity Matrix

The identity matrix plays a similar role as the number 1. Multiplying any number by

1 gives back that number. In matrix algebra, pre or post multiplying a matrix A by

a conformable identity matrix gives back the matrix A. To illustrate, let

Â· Â¸

10

I=

01

denote the 2 dimensional identity matrix and let

Â· Â¸

a11 a12

A=

a21 a22

denote an arbitrary 2 Ã— 2 matrix. Then

Â· Â¸Â· Â¸

10 a11 a12

IÂ·A = Â·

01 a21 a22

Â· Â¸

a11 a12

= =A

a21 a22

and

Â· Â¸Â· Â¸

a11 a12 10

AÂ·I = Â·

a21 a22 01

Â· Â¸

a11 a12

= = A.

a21 a22

1.3 Inverse Matrix

To be completed.

1.4 Representing Summation Using Vector Notation

Consider the sum

n

X

xk = x1 + Â· Â· Â· + xk.

k=1

Let x = (x1 , . . . , xn )0 be an n Ã— 1 vector and 1 = (1, . . . , 1)0 be an n Ã— 1 vector of

ones. Then

ï£®ï£¹

1 n

X

Â£ Â¤ ï£¯.ï£º

0

x 1 = x1 . . . xn Â· ï£° . ï£» = x1 + Â· Â· Â· + xk = xk

.

1 k=1

4

and ï£®

ï£¹

x1 n

X

Â£ Â¤ï£¯.ï£º

0

1 x = 1 . . . 1 Â· ï£° . ï£» = x1 + Â· Â· Â· + xn = xk .

.

xn k=1

Next, consider the sum of squared x values

n

X

x2 = x2 + Â· Â· Â· + x2 .

k 1 n

k=1

This sum can be conveniently represented as

ï£® ï£¹

x

Â¤ ï£¯ .1 n

X

Â£ ï£º

x0 x = x1 . . . xn Â· ï£° . 2 2

x2 .

ï£» = x1 + Â· Â· Â· + xn =

. k

xn k=1

Last, consider the sum of cross products

n

X

xk yk = x1 y1 + Â· Â· Â· xn yn .

k=1

This sum can be compactly represented by

ï£® ï£¹

y

Â¤ ï£¯ .1 ï£º Xn

Â£

0

x y = x1 . . . xn Â· ï£° . ï£» = x1 y1 + Â· Â· Â· xn yn = xk yk .

.

yn k=1

Note that x0 y = y0 x.

1.5 Representing Systems of Linear Equations Using Matrix

Algebra

Consider the system of two linear equations

(1)

x+y = 1

2x âˆ’ y = 1 (2)

which is illustrated in Figure xxx. Equations (1) and (2) represent two straight lines

which intersect at the point x = 2 and y = 1 . This point of intersection is determined

3 3

by solving for the values of x and y such that x + y = 2x âˆ’ y 1 .

1

Soving for x gives x = 2y. Substituting this value into the equation x + y = 1 gives 2y + y = 1

and solving for y gives y = 1/3. Solving for x then gives x = 2/3.

5

The two linear equations can be written in matrix form as

Â· Â¸Â· Â¸ Â· Â¸

11 x 1

=

2 âˆ’1 y 1

or

AÂ·z=b

where Â· Â¸ Â· Â¸ Â· Â¸

11 x 1

and b =

A= , z= .

2 âˆ’1 y 1

If there was a (2 Ã— 2) matrix B, with elements bij , such that B Â· A = I, where I

is the (2 Ã— 2) identity matrix, then we could solve for the elements in z as follows. In

the equation A Â· z = b, pre-multiply both sides by B to give

BÂ·AÂ·z = BÂ·b

=â‡’ I Â· z = B Â· b

=â‡’ z = B Â· b

or Â· Â¸ Â· Â¸Â· Â¸ Â· Â¸

b11 Â· 1 + b12 Â· 1

x b11 b12 1

= =

b21 Â· 1 + b22 Â· 1

y b21 b22 1

If such a matrix B exists it is called the inverse of A and is denoted Aâˆ’1 . In-

tuitively, the inverse matrix Aâˆ’1 plays a similar role as the inverse of a number.

1

Suppose a is a number; e.g., a = 2. Then we know that a Â· a = aâˆ’1 a = 1. Similarly,

in matrix algebra Aâˆ’1 A = I where I is the identity matrix. Next, consider solving

1

the equation ax = 1. By simple division we have that x = a x = aâˆ’1 x. Similarly, in

matrix algebra if we want to solve the system of equation Ax = b we multiply by

Aâˆ’1 and get x = Aâˆ’1 b.

Using B = Aâˆ’1 , we may express the solution for z as

z = Aâˆ’1 b.

As long as we can determine the elements in Aâˆ’1 then we can solve for the values of

x and y in the vector z. Since the system of linear equations has a solution as long as

the two lines intersect, we can determine the elements in Aâˆ’1 provided the two lines

are not parallel.

nding the elements of Aâˆ’1 and typical

There are general numerical algorithms for &

spreadsheet programs like Excel have these algorithms available. However, if A is a

(2 Ã— 2) matrix then there is a simple formula for Aâˆ’1 . Let A be a (2 Ã— 2) matrix such

that Â· Â¸

a11 a12

A= .

a21 a22

6

Then Â· Â¸

1 a22 âˆ’a12

Aâˆ’1 = .

âˆ’a21 a11

a11 a22 âˆ’ a21 a12

By brute force matrix multiplication we can verify this formula

Â· Â¸Â· Â¸

1 a22 âˆ’a12 a11 a12

Aâˆ’1 A =

a11 a22 âˆ’ a21 a12 âˆ’a21 a11 a21 a22

Â· Â¸

1 a22 a11 âˆ’ a12 a21 a22 a12 âˆ’ a12 a22

=

a11 a22 âˆ’ a21 a12 âˆ’a21 a11 + a11 a21 âˆ’a21 a12 + a11 a22

Â· Â¸

1 a22 a11 âˆ’ a12 a21 0

=

âˆ’a21 a12 + a11 a22

0

a11 a22 âˆ’ a21 a12

Â· a22 a11 âˆ’a12 a21 Â¸

0

a11 a22 âˆ’a21 a12

= âˆ’a21 a12 +a11 a22

0 a11 a22 âˆ’a21 a12

Â· Â¸

10

= .

01

Let s apply the above rule to & the inverse of A in our example:

nd

Â· Â¸ Â·1 1 Â¸

1 âˆ’1 âˆ’1

Aâˆ’1 = = 3 âˆ’1 . 3

2

âˆ’1 âˆ’ 2 âˆ’2 1 3 3

Notice that Â· Â¸Â· Â¸ Â· Â¸

1 1

11 10

Aâˆ’1 A = 3 3 = .

2 âˆ’1

2 âˆ’1 01

3 3

Our solution for z is then

z = Â· âˆ’1 b Â¸ Â· Â¸

A

1 1

1

3 3

= 2 âˆ’1

1

3

Â¸3 Â· Â¸

Â·2

x

3

= =

1

y

3

so that x = 2 and y = 1 .

3 3

In general, if we have n linear equations in n unknown variables we may write the

system of equations as

a11 x1 + a12 x2 + Â· Â· Â· + a1n xn = b1

a21 x1 + a22 x2 + Â· Â· Â· + a2n xn = b2

.

.=. .

. .

an1 x1 + an2 x2 + Â· Â· Â· + ann xn = bn

7

which we may then express in matrix form as

ï£® ï£¹ï£® ï£¹ ï£® ï£¹

a12 Â· Â· Â· a1n

a11 x1 b1

ï£¯ a21 a22 Â· Â· Â· a2n ï£º ï£¯ ï£ºï£¯ ï£º

x2 b2

ï£¯ ï£ºï£¯ ï£ºï£¯ ï£º

ï£¯. . ï£ºï£¯ ï£º=ï£¯ ï£º

. .

ï£°. . ï£»ï£° . .

ï£»ï£° ï£»

. . . .

an2 Â· Â· Â· ann

an1 xn bn

or

A Â· x = b.

(nÃ—1) (nÃ—1)

(nÃ—n)

The solution to the system of equations is given by

x = Aâˆ’1 b

where Aâˆ’1 A = I and I is the (n Ã— n) identity matrix. If the number of equations is

greater than two, then we generally use numerical algorithms to & the elements in

nd

Aâˆ’1 .

2 Further Reading

Excellent treatments of portfolio theory using matrix algebra are given in Ingersol

(1987), Huang and Litzenberger (1988) and Campbell, Lo and MacKinlay (1996).

3 Problems

To be completed

References

[1] Campbell, J.Y., Lo, A.W., and MacKinlay, A.C. (1997). The Econometrics of

Financial Markets. Priceton, New Jersey: Princeton University Press.

[2] Huang, C.-F., and Litzenbeger, R.H. (1988). Foundations for Financial Eco-

nomics. New York: North-Holland.

[3] Ingersoll, J.E. (1987). Theory of Financial Decision Making. Totowa, New Jersey:

Rowman & Little& eld.

8