ENGG2013 Unit 7Non-singular matrix

and Gauss-Jordan eliminationJan, 2011.

Outline

• Matrix arithmetic– Matrix addition, multiplication

• Non-singular matrix• Gauss-Jordan elimination

kshum ENGG2013 2

The love function: a normal case

kshum ENGG2013 3

Boy 1

Boy 2

Boy 3

Boy 4

Boy 5

Girl A

Girl B

Girl C

Girl D

Girl E

Function LDomain Range

Boy 1

Boy 2

Boy 3

Boy 4

Boy 5

Girl A

Girl B

Girl C

Girl D

Girl E

DomainRange

L(Boy 1) = Girl A, but L’(Girl A) = Boy 4.

Function L’

The love function: a utopian case

kshum ENGG2013 4

Boy 1

Boy 2

Boy 3

Boy 4

Boy 5

Girl A

Girl B

Girl C

Girl D

Girl E

Function LDomain Range

Boy 1

Boy 2

Boy 3

Boy 4

Boy 5

Girl A

Girl B

Girl C

Girl D

Girl E

Function L’Domain Range

This function L’ is the inverse of L

The love function: no inverse

kshum ENGG2013 5

Boy 1

Boy 2

Boy 3

Boy 4

Boy 5

Girl A

Girl B

Girl C

Girl D

Girl E

Function LDomain Range

Boy 1

Boy 2

Boy 3

Boy 4

Boy 5

Girl A

Girl B

Girl C

Girl D

Girl E

Domain Range

This function L has no inverse

This is not a function

Undo-able

kshum ENGG2013 6

Multiplied by

Rotate 90 degrees clockwise

Multiplied by

Rotate 90 degrees counter-clockwise

A matrix which represents a reversibleprocess is called invertible or non-singular.

Objectives

• How to determine whether a matrix is invertible?

• If a matrix is invertible, how to find the corresponding inverse matrix?

kshum ENGG2013 7

MATRIX ALGEBRA

kshum ENGG2013 8

Matrix equality

• Two matrices are said to be equal if1. They have the same number of rows and the same number

of columns (i.e. same size).2. The corresponding entry are identical.

kshum ENGG2013 9

Matrix addition and scalar multiplication

• We can add two matrices if they have the same size

• To multiply a matrix by a real number, we just multiply all entries in the matrix by that number.

kshum ENGG2013 10

Matrix multiplication

• Given an mn matrix A and a pq matrix B, their product AB is defined if n=p.

• If n = p, we define their product, say C = AB, by computing the (i,j)-entry in C as the dot product of the i-th row of A and the j-th row of B.

kshum ENGG2013 11

mnpq

m q

Examples

kshum ENGG2013 12

is undefined.

is undefined.

Square matrix• A matrix with equal number of columns and rows is

called a square matrix.• For square matrices of the same size, we can freely

multiply them without worrying whether the product is well-defined or not.– Because multiplication is always well-defined in this case.

• The entries with the same column and row index are called the diagonal entries.– For example:

kshum ENGG2013 13

Compatibility with function composition

kshum ENGG2013 14

Multiplied by

Multiplied by

Multiplied by

Order does matter in multiplication

kshum ENGG2013 15

Multiplied by

Rotate 90 degrees

Multiplied by

Reflection around x-axis

Multiplied by

Reflection around x-axis

Multiplied by

Rotate 90 degrees

Are they the same?

Non-commutativity

• For real numbers, we have 35 = 53.– Multiplication of real numbers is commutative.

• For matrices, in general AB BA.– Multiplication of matrices is non-commutative.– For example

kshum ENGG2013 16

Associativity

• For real numbers, we have (34)5 = 3(45).– Multiplication of real numbers is associative.

• For any three matrices A, B, C, it is always true that (AB)C = A(BC), provided that the multiplications are well-defined.– Multiplication of matrices is associative.

kshum ENGG2013 17

INVERTIBLE MATRIX

kshum ENGG2013 18

Identity matrix• A square matrix whose diagonal entries are all one, and off-diagonal entries are all zero,

is called an identity matrix.

• We usually use capital letter I for identity matrix, or add a subscript and write In if we want to stress that the size is nn.

kshum ENGG2013 19

Multiplication by identity matrix

• Identity matrix is like a do-nothing process.– There is no change after multiplication by the

identity matrix

• IA = A for any A.• BI = B for any B.

kshum ENGG2013 20

Multiplied by

is trivial

Invertible matrix• Given an nn matrix A, if we can find a matrix A’, such that

then A is said to be invertible, or non-singular.• This matrix A’ is called an inverse of A.

kshum ENGG2013 21

Multiplied by

A

Multiplied by

A’

Multiplied by

In

Example

kshum ENGG2013 22

Multiplied by

Rotate 90 CW

Multiplied by

Rotate 90 CCW

implies is invertible.

Matrix inverse may not exist

• If matrix A induces a many-to-one mapping, then we cannot hope for any inverse.

kshum ENGG2013 23

has no inverse

Naïve method for computing matrix inverse

• Consider

• Want to find A’ such that A A’= I• Solve for p, q, r, s in

kshum ENGG2013 24

Uniqueness of matrix inverse

• Before we discuss how to compute matrix inverse, we first show there is at most one A’ such that A A’ = A’ A = I.

• Suppose on the contrary that there is another matrix A’’ such that A A’’ = A’’ A = I.

• We want to prove that A’ = A’’.

kshum ENGG2013 25

Proof of uniqueness

kshum ENGG2013 26

Defining property of A’’

Multiply by A’ from the left

I times anything is the same thing

Matrix multiplication is associative

Defining property of A’

I times anything is the same thing

Notation

• Since the matrix inverse (if exists) is unique, we use the symbol A-1 to represent the unique matrix which satisfies

• We say that A-1 is the inverse of A.

kshum ENGG2013 27

A convenient fact

• To check that a matrix B is the inverse of A, it is sufficient to check either 1. BA = I, or2. AB = I.

• It can be proved that (1) implies (2), and (2) implies (1).– The details is left as exercise.

kshum ENGG2013 28

GAUSS-JORDAN ELIMINATION

kshum ENGG2013 29

Row operation using matrix

• Recall that there are three kind of elementary row operations1. Row exchange2. Multiply a row by a non-zero constant3. Replace a row by the sum of itself and a

constant multiple of another row.

• We can perform elementary row operation by matrix multiplication (from the left).

• All three kinds of operation are invertible.kshum ENGG2013 30

Row exchange

• Example: exchange row 2 and row 3

Multiply the same matrix from the left again, we get back the original matrix.

kshum ENGG2013 31

Multiply a row by a constant

• Multiply the first row by -1.

Multiply the same matrix from the left again, we get back the original matrix.

kshum ENGG2013 32

Row replacement

• Add the first row to the second row

Multiply by another matrix from the left to undo

kshum ENGG2013 33

Elementary matrix (I)• Three types of elementary matrices

1. Exchange row i and row j

kshum ENGG2013 34

Row i

Row j

Col

.

jCol

.

i

Elementary matrix (II)2. Multiply row i by m

kshum ENGG2013 35

Row i

Col

.

i

Elementary matrix (III)3. Add s times row i to row j

kshum ENGG2013 36

Row i

Row j

Col

.

jCol

.

i

Row reduction

• A series of row reductions is the same as multiplying from the left a series of elementary matrices.

kshum ENGG2013 37

…

E1, E2, E3, … are elementary matrices.

If we can row reduce to identity

• Then A is non-singular, or invertible.

kshum ENGG2013 38

(Matrixmultiplication isassociative)

Gauss-Jordan elimination

• It is convenient to append an identity matrix to the right

• We can interpret it asIf we can row reduce A to the identity by a series of

row operationsthen we can apply the same series of row operations to I and obtain the inverse of A.

kshum ENGG2013 39

Algorithm

• Input: an nn matrix A.• Create an n 2n matrix M

– The left half is A– The right half is In

• Try to reduce the expanded matrix M such that the left half is equal to In.

• If succeed, the right half of M is the inverse of A.• If you cannot reduce the left half of M to , then A

is not invertible, a.k.a. singular.

kshum ENGG2013 40

Example

• Find the inverse of

1. Create a 36 matrix 2. After some row reductions

we get

• Answer: kshum ENGG2013 41