8/9/2019 Summary of Matrix
1/25
Multiplication of Matrices
Before we give the formal definition of how to multiply two matrices, we will discuss anexample from a real life situation. Consider a city with two kinds of population: the innercity population and the suburb population. We assume that every year 40% of the inner city
population moves to the suburbs, while 30% of the suburb population moves to the innerpart of the city. Let I(resp. S) be the initial population of the inner city (resp. the suburban
area). So after one year, the population of the inner part is
0.6 I+ 0.3 S
while the population of the suburbs is
0.4I
+ 0.7S
After two years, the population of the inner city is
0.6 (0.6 I+ 0.3 S) + 0.3 (0.4 I+ 0.7 S)
and the suburban population is given by
0.4 (0.6 I+ 0.3 S) + 0.7(0.4 I+ 0.7 S)
Is there a nice way of representing the two populations after a certain number of years? Let
us show how matrices may be helpful to answer this question. Let us represent the twopopulations in one table (meaning a column object with two entries):
So after one year the table which gives the two populations is
If we consider the following rule (the product of two matrices)
8/9/2019 Summary of Matrix
2/25
then the populations after one year are given by the formula
After two years the populations are
Combining this formula with the above result, we get
In other words, we have
In fact, we do not need to have two matrices of the same size to multiply them. Above, wedid multiply a (2x2) matrix with a (2x1) matrix (which gave a (2x1) matrix). In fact, the
general rule says that in order to perform the multiplicationAB, whereA is a (mxn) matrixandB a (kxl) matrix, then we must have n=k. The result will be a (mxl) matrix. For
example, we have
Remember that though we were able to perform the above multiplication, it is not possibleto perform the multiplication
So we have to be very careful about multiplying matrices. Sentences like "multiply the two
matricesA andB" do not make sense. You must know which of the two matrices will be tothe right (of your multiplication) and which one will be to the left; in other words, we have
8/9/2019 Summary of Matrix
3/25
to know whether we are asked to perform or . Even if bothmultiplications do make sense (as in the case of square matrices with the same size), we
still have to be very careful. Indeed, consider the two matrices
We have
and
So what is the conclusion behind this example? The matrix multiplication is notcommutative, the order in which matrices are multiplied is important. In fact, this little
setback is a major problem in playing around with matrices. This is something that youmust always be careful with. Let us show you another setback. We have
the product of two non-zero matrices may be equal to the zero-matrix.
8/9/2019 Summary of Matrix
4/25
Algebraic Properties of Matrix Operations
In this page, we give some general results about the three operations: addition,multiplication, and multiplication with numbers, called scalar multiplication.
From now on, we will not write (mxn) but mxn.
Properties involving Addition. LetA,B, and Cbe mxn matrices. We have
1.
A+B =B+A2.
(A+B)+C=A + (B+C)
3.
where is the mxn zero-matrix (all its entries are equal to 0);4.
if and only ifB = -A.
Properties involving Multiplication.
1.
LetA,B, and Cbe three matrices. If you can perform the productsAB, (AB)C,BC,andA(BC), then we have(AB)C=A (BC)
Note, for example, that ifA is 2x3,B is 3x3, and Cis 3x1, then the above productsare possible (in this case, (AB)Cis 2x1 matrix).
2.
If and are numbers, andA is a matrix, then we have
3.
If is a number, andA andB are two matrices such that the product is
possible, then we have
8/9/2019 Summary of Matrix
5/25
4.
IfA is an nxm matrix and the mxk zero-matrix, then
Note that is the nxk zero-matrix. So if n is different from m, the two zero-matrices are different.
Properties involving Addition and Multiplication.
1.LetA,B, and Cbe three matrices. If you can perform the appropriate products, then
we have(A+B)C=AC+BC
andA(B+C) =AB +AC
2.
If and are numbers,A andB are matrices, then we have
and
Example. Consider the matrices
Evaluate (AB)CandA(BC). Check that you get the same matrix.
Answer. We have
8/9/2019 Summary of Matrix
6/25
so
On the other hand, we have
so
Example. Consider the matrices
It is easy to check that
and
These two formulas are called linear combinations. More on linear combinations will be
discussed on a different page.
8/9/2019 Summary of Matrix
7/25
We have seen that matrix multiplication is different from normal multiplication (between
numbers). Are there some similarities? For example, is there a matrix which plays a similarrole as the number 1? The answer is yes. Indeed, consider the nxn matrix
In particular, we have
The matrix In has similar behavior as the number 1. Indeed, for any nxn matrixA, we have
A In = InA =A
The matrix In is called the Identity Matrix of order n.
Example. Consider the matrices
Then it is easy to check that
The identity matrix behaves like the number 1 not only among the matrices of the form
nxn. Indeed, for any nxm matrixA, we have
8/9/2019 Summary of Matrix
8/25
In particular, we have
8/9/2019 Summary of Matrix
9/25
Invertible Matrices
Invertible matrices are very important in many areas of science. For example, decrypting acoded message uses invertible matrices. The problem of finding the inverse of a matrix willbe discussed in a different page .
Definition. An matrixA is called nonsingular orinvertible iff there exists an
matrixB such that
where In is the identity matrix. The matrixB is called the inverse matrix ofA.
Example. Let
One may easily check that
HenceA is invertible andB is its inverse.
Notation. A common notation for the inverse of a matrixA isA-1
. So
Example. Find the inverse of
8/9/2019 Summary of Matrix
10/25
Write
Since
we get
Easy algebraic manipulations give
or
The inverse matrix is unique when it exists. So ifA is invertible, thenA-1
is also invertible
and
The following basic property is very important:
8/9/2019 Summary of Matrix
11/25
IfA andB are invertible matrices, then is also invertible and
Remark. In the definition of an invertible matrixA, we used both and to beequal to the identity matrix. In fact, we need only one of the two. In other words, for a
matrixA, if there exists a matrixB such that , thenA is invertible andB =A-1
.
8/9/2019 Summary of Matrix
12/25
Elementary Operations for Matrices
Elementary operations for matrices play a crucial role in finding the inverse or solvinglinear systems. They may also be used for other calculations. On this page, we will discussthese type of operations. Before we define an elementary operation, recall that to an nxm
matrixA, we can associate n rows and m columns. For example, consider the matrix
Its rows are
Its columns are
Let us consider the matrix transpose ofA
Its rows are
As we can see, the transpose of the columns ofA are the rows ofAT. So the transpose
operation interchanges the rows and the columns of a matrix. Therefore many techniqueswhich are developed for rows may be easily translated to columns via the transpose
operation. Thus, we will only discuss elementary row operations, but the reader may easilyadapt these to columns.
8/9/2019 Summary of Matrix
13/25
Elementary Row Operations.
1.Interchange two rows.
2.
Multiply a row with a nonzero number.3.Add a row to another one multiplied by a number.
Definition. Two matrices are row equivalent if and only if one may be obtained from the
other one via elementary row operations.
Example. Show that the two matrices
are row equivalent.
Answer. We start withA. If we keep the second row and add the first to the second, we get
We keep the first row. Then we subtract the first row from the second one multiplied by 3.We get
We keep the first row and subtract the first row from the second one. We get
which is the matrixB. ThereforeA andB are row equivalent.
One powerful use of elementary operations consists in finding solutions to linear systemsand the inverse of a matrix. This happens via Echelon Form and Gauss-Jordan
Elimination. In order to appreciate these two techniques, we need to discuss when a matrixis row elementary equivalent to a triangular matrix. Let us illustrate this with an example.
8/9/2019 Summary of Matrix
14/25
Example. Consider the matrix
First we will transform the first column via elementary row operations into one with the topnumber equal to 1 and the bottom ones equal 0. Indeed, if we interchange the first row with
the last one, we get
Next, we keep the first and last rows. And we subtract the first one multiplied by 2 from thesecond one. We get
We are almost there. Looking at this matrix, we see that we can still take care of the 1(from the last row) under the -2. Indeed, if we keep the first two rows and add the second
one to the last one multiplied by 2, we get
We can't do more. Indeed, we stop the process whenever we have a matrix which satisfiesthe following conditions
1.any row consisting of zeros is below any row that contains at least one nonzero
number;2.
the first (from left to right) nonzero entry of any row is to the left of the firstnonzero entry of any lower row.
Now if we make sure that the first nonzero entry of every row is 1, we get a matrix in row
echelon form. For example, the matrix above is not in echelon form. But if we divide thesecond row by -2, we get
8/9/2019 Summary of Matrix
15/25
This matrix is in echelon form.
An application of this, namely to solve linear systems via Gaussian elimination may be
found on another page.
8/9/2019 Summary of Matrix
16/25
Systems of Linear Equations: Gaussian
Elimination
It is quite hard to solve non-linear systems of equations, while linear systems are quite easy
to study. There are numerical techniques which help to approximate nonlinear systems withlinear ones in the hope that the solutions of the linear systems are close enough to the
solutions of the nonlinear systems. We will not discuss this here. Instead, we will focus ourattention on linear systems.
For the sake of simplicity, we will restrict ourselves to three, at most four, unknowns. The
reader interested in the case of more unknowns may easily extend the following ideas.
Definition. The equation
a x + b y + c z + d w = h
where a, b, c, d, and h are known numbers, whilex,y,z, and w are unknown numbers, iscalled a linear equation. Ifh =0, the linear equation is said to be homogeneous. A linear
system is a set of linear equations and a homogeneous linear system is a set ofhomogeneous linear equations.
For example,
and
are linear systems, while
is a nonlinear system (because ofy2). The system
8/9/2019 Summary of Matrix
17/25
is an homogeneous linear system.
Matrix Representation of a Linear System
Matrices are helpful in rewriting a linear system in a very simple form. The algebraic
properties of matrices may then be used to solve systems. First, consider the linear system
Set the matrices
Using matrix multiplications, we can rewrite the linear system above as the matrix equation
As you can see this is far nicer than the equations. But sometimes it is worth to solve thesystem directly without going through the matrix form. The matrixA is called the matrix
coefficient of the linear system. The matrix Cis called the nonhomogeneous term. When
, the linear system is homogeneous. The matrix X is the unknown matrix. Itsentries are the unknowns of the linear system. The augmented matrix associated with the
system is the matrix [A|C], where
8/9/2019 Summary of Matrix
18/25
In general if the linear system has n equations with m unknowns, then the matrix coefficient
will be a nxm matrix and the augmented matrix an nx(m+1) matrix. Now we turn ourattention to the solutions of a system.
Definition. Two linear systems with n unknowns are said to be equivalent if and only if
they have the same set of solutions.
This definition is important since the idea behind solving a system is to find an equivalent
system which is easy to solve. You may wonder how we will come up with such system?Easy, we do that through elementary operations. Indeed, it is clear that if we interchange
two equations, the new system is still equivalent to the old one. If we multiply an equationwith a nonzero number, we obtain a new system still equivalent to old one. And finally
replacing one equation with the sum of two equations, we again obtain an equivalentsystem. These operations are called elementary operations on systems. Let us see how it
works in a particular case.
E
xample. Consider the linear system
The idea is to keep the first equation and work on the last two. In doing that, we will try to
kill one of the unknowns and solve for the other two. For example, if we keep the first and
second equation, and subtract the first one from the last one, we get the equivalent system
Next we keep the first and the last equation, and we subtract the first from the second. Weget the equivalent system
Now we focus on the second and the third equation. We repeat the same procedure. Try to
kill one of the two unknowns (y orz). Indeed, we keep the first and second equation, andwe add the second to the third after multiplying it by 3. We get
8/9/2019 Summary of Matrix
19/25
This obviously impliesz = -2. From the second equation, we gety = -2, and finally from thefirst equation we getx = 4. Therefore the linear system has one solution
Going from the last equation to the first while solving for the unknowns is called
backsolving.
Keep in mind that linear systems for which the matrix coefficient is upper-triangular areeasy to solve. This is particularly true, if the matrix is in echelon form. So the trick is to
perform elementary operations to transform the initial linear system into another one forwhich the coefficient matrix is in echelon form.
Using our knowledge about matrices, is there anyway we can rewrite what we did above inmatrix form which will make our notation (or representation) easier? Indeed, consider the
augmented matrix
Let us perform some elementary row operations on this matrix. Indeed, if we keep the first
and second row, and subtract the first one from the last one we get
Next we keep the first and the last rows, and we subtract the first from the second. We get
Then we keep the first and second row, and we add the second to the third after multiplyingit by 3 to get
8/9/2019 Summary of Matrix
20/25
This is a triangular matrix which is not in echelon form. The linear system for which thismatrix is an augmented one is
As you can see we obtained the same system as before. In fact, we followed the same
elementary operations performed above. In every step the new matrix was exactly the
augmented matrix associated to the new system. This shows that instead of writing thesystems over and over again, it is easy to play around with the elementary row operationsand once we obtain a triangular matrix, write the associated linear system and then solve it.
This is known as Gaussian Elimination. Let us summarize the procedure:
Gaussian Elimination. Consider a linear system.
1.Construct the augmented matrix for the system;
2.Use elementary row operations to transform the augmented matrix into a triangular
one;3.
Write down the new linear system for which the triangular matrix is the associated
augmented matrix;
4.Solve the new system. You may need to assign some parametric values to someunknowns, and then apply the method of back substitution to solve the new system.
Example. Solve the following system via Gaussian elimination
The augmented matrix is
8/9/2019 Summary of Matrix
21/25
We use elementary row operations to transform this matrix into a triangular one. We keepthe first row and use it to produce all zeros elsewhere in the first column. We have
Next we keep the first and second row and try to have zeros in the second column. We get
Next we keep the first three rows. We add the last one to the third to get
This is a triangular matrix. Its associated system is
Clearly we have v = 1. Setz=s and w=t, then we have
The first equation implies
8/9/2019 Summary of Matrix
22/25
x = 2 + y + z - w - v.Using algebraic manipulations, we get
x = - - s - t.
Putting all the stuff together, we have
Example. Use Gaussian elimination to solve the linear system
The associated augmented matrix is
We keep the first row and subtract the first row multiplied by 2 from the second row. We
get
This is a triangular matrix. The associated system is
8/9/2019 Summary of Matrix
23/25
Clearly the second equation implies that this system has no solution. Therefore this linearsystem has no solution.
Definition. A linear system is called inconsistent or overdetermined if it does not have a
solution. In other words, the set of solutions is empty. Otherwise the linear system is calledconsistent.
Following the example above, we see that if we perform elementary row operations on the
augmented matrix of the system and get a matrix with one of its rows equal to
, where , then the system is inconsistent.
8/9/2019 Summary of Matrix
24/25
Application of Determinant to Systems:
Cramer's Rule
We have seen that determinant may be useful in finding the inverse of a nonsingular matrix.
We can use these findings in solving linear systems for which the matrix coefficient isnonsingular (or invertible).
Consider the linear system (in matrix form)
A X=B
whereA is the matrix coefficient,B the nonhomogeneous term, andX
the unknowncolumn-matrix. We have:
Theorem. The linear systemAX=B has a unique solution if and only ifA is invertible. In
this case, the solution is given by the so-called Cramer's formulas:
wherexi are the unknowns of the system or the entries ofX, and the matrixAi is obtained
fromA by replacing the ith column by the columnB. In other words, we have
where the bi are the entries ofB.
In particular, if the linear systemAX=B is homogeneous, meaning , then ifA is
invertible, the only solution is the trivial one, that is . So if we are looking for anonzero solution to the system, the matrix coefficientA must be singular or noninvertible.
We also know that this will happen if and only if . This is an importantresult.
Example. Solve the linear system
8/9/2019 Summary of Matrix
25/25
Answer. First note that
which implies that the matrix coefficient is invertible. So we may use the Cramer's
formulas. We have
We leave the details to the reader to find
Note that it is easy to see thatz=0. Indeed, the determinant which givesz has two identicalrows (the first and the last). We do encourage you to check that the values found forx,y,
andz are indeed the solution to the given system.
Remark. Remember that Cramer's formulas are only valid for linear systems with an
invertible matrix coefficient.