Introduction to SAS Vectors and Matrices · PDF fileIntroduction to SAS. Vectors and Matrices...

transcript

Lecture #2 - 8/31/2005 Slide 1 of 55

Introduction to SASVectors and Matrices

Lecture 2

August 31, 2005Multivariate Analysis

Overview

● Today’s Lecture

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 2 of 55

Today’s Lecture

■ Introduction to SAS.

■ Vectors and Matrices (Supplement 2A - augmented withSAS proc iml).

Lecture #2 - 8/31/2005 Slide 3 of 55

SAS■ To start SAS, on a Windows PC, go to Start...All Programs...The SAS

System...The SAS System for Windows V8.

Lecture #2 - 8/31/2005 Slide 4 of 55

Main Program Window■ The SAS program looks like this (some helpful commands are shown with

the arrows):

Overview

Introduction to SAS

● Main Program Window

● SAS Editor

● SAS Code

● First SAS Program

● Run

● SAS Data Libraries

● SAS Procedures

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 5 of 55

SAS Editor■ To run SAS you must create a file of SAS code, which the

SAS processor reads and uses to run the program.

■ Simply type your SAS code into the Program Editor window.

■ For our example today, we will create (and save to) a newSAS code file, so to do that be sure to have your curserinside of the editor window and go to File...Save...

■ SAS code files usually end with the extension *.sas.

Overview

Introduction to SAS

● SAS Editor

● SAS Code

● Run

● SAS Procedures

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 6 of 55

SAS Code Basics■ For use in day-to-day statistical applications, SAS code

consists of two components:

◆ Data steps (where data input usually happens.

◆ Proc steps (where statistical analyses usually happen.

■ General exceptions to these rules exist.

■ All statements terminate with a semicolon (this is usuallywhere errors can occur).

■ Commented code can begin with:

◆ An asterisk (*) for single lines - terminated with asemicolon.

◆ /* for multiple lines, terminated with an ending */.

■ Enough talk...how about an example? Type the following intothe SAS Editor:

Lecture #2 - 8/31/2005 Slide 7 of 55

First SAS Programtitle ’First SAS Program’;data example1;input id gender $ age;cards;1 F 232 M 203 F 184 F .5 M 196 M 217 M 258 F 249 F 2010 M 22;

proc print;var id gender age;run;

Overview

Introduction to SAS

● SAS Editor

● SAS Code

● Run

● SAS Procedures

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 8 of 55

First SAS ProgramNotice the color scheme of the SAS Enhanced Editor (note: ifyou do not see color, do not panic, you may not have theEnhanced Editor installed).

■ Items in light blue are command words (like “input” or “var”.

■ Items in dark blue are procedural words (like “proc,” “data,” or“run”).

Overview

Introduction to SAS

● SAS Editor

● SAS Code

● Run

● SAS Procedures

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 9 of 55

Run the SAS Program■ To run the program you just entered, press the running man

icon at the center of the top of the SAS main program.

■ Once you press the “run” button, the log window will becomeactive, giving you information about the program as itexecutes.

■ In this window you will see errors (in red), or warnings (ingreen I think).

■ As multivariate progresses, you will become aware ofinstances where warnings will be present because ofproblems in your analysis.

■ Also now appearing is a new window called output...this iswhere the output of the procedure that was just run isdisplayed.

Overview

Introduction to SAS

● SAS Editor

● SAS Code

● Run

● SAS Procedures

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 10 of 55

SAS Data Libraries■ The data set you just entered is now part of a SAS data

library that can be referenced at any point during theremainder of the program.

■ The default SAS data library is called “work,” and can beaccessed by clicking through the explorer window on the lefthand side of the program.

■ Double click on Libraries...Work...Example1 and you will seethe data displayed in a grid.

■ To familiarize you with SAS, here are some handouts (alsoavailable on the BlackBoard Site)...

Lecture #2 - 8/31/2005 Slide 11 of 55

SAS Procedures■ The bulk of the statistical work done in SAS is through procedure statements.

■ Proc statements follow a flexible syntax that typically has the following:

proc -statement_name- data=-data_name- [options];var [included variables];[options];run;

■ The names and options are all found in the SAS manual, which is freely(shhh) available online at:http://www.id.unizh.ch/software/unix/statmath/sas/sasdoc/main.htm

Lecture #2 - 8/31/2005 Slide 12 of 55

SAS Procedures Example■ Using the Example1 data set, type the following into the editor:

proc sort data=example1;by gender;run;

proc univariate data=example1 plots;by gender;var age;run;

■ The univariate procedure produces univariate statistics, the manual entry canbe at found:

http://www.id.unizh.ch/software/unix/statmath/sas/sasdoc/proc/z0146802.htm

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Matrix Computing

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 13 of 55

Matrix Introduction

■ Imagine you are interested in studying the culturaldifferences in student achievement in high school.

■ Your study gets IRB approval and you work hard to getparental approval.

■ You go into school and collect data using many differentinstruments.

■ You then input the data into your favorite stat package, likeSAS (or MS Excel).

■ How do you think the data is stored?

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 14 of 55

Why Learn Matrix Algebra?

■ Nearly all multivariate techniques are founded in matrixalgebra.

■ Often times when statistics break down, the cause of thefailure can be traced through matrix procedures.

■ If you are trying to apply a new method, most of the technicalstatistical literature uses matrix algebra assuming a basic toadvanced knowledge of matrices - you may need to readthese articles.

■ Have you seen:◆ (X′X)−1X′y?◆ ΛΦΛ

′ + Ψ?◆ F1F2F3LF−1

3 F−12 P(F−1

2 )′(F−13 )′L′F′

1 + U2?

■ Warning: using matrix algebra lingo is a great way to endconversations or break up parties.

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 15 of 55

Definitions

■ We begin this class with some general definitions (fromdictionary.com):

◆ Matrix:

1. A rectangular array of numeric or algebraic quantitiessubject to mathematical operations.

2. The substrate on or within which a fungus grows.

◆ Algebra:

1. A branch of mathematics in which symbols, usuallyletters of the alphabet, represent numbers or membersof a specified set and are used to represent quantitiesand to express general relationships that hold for allmembers of the set.

2. A set together with a pair of binary operations definedon the set. Usually, the set and the operations includean identity element, and the operations arecommutative or associative.

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 16 of 55

Proc IML

■ To help demonstrate the topics we will discuss today, I will beshowing examples in SAS proc iml.

■ The Interactive Matrix Language (IML) is a scientificcomputing package in SAS that typically used forcomplicated statistical routines.

■ Of course, other matrix programs exist - for many statisticalapplications MATLAB is very useful.

■ SPSS and SAS both have matrix computing capabilities, but(in my opinion) neither are as efficient, as user friendly, or asflexible as MATLAB.

◆ It is better to leave most of the statistical computing to thecomputer scientists.

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 17 of 55

Proc IML Basics

■ Proc IML is a proc step in SAS that runs without needing touse a preliminary data step.

■ To use IML, make sure the following are placed in a SAScode file.

proc iml;

reset print;

■ The “reset print;” line makes every result get printed in theoutput window.

■ The IML code will go between the “reset print;” and the “quit;”

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 18 of 55

Matrices

■ Away from the definition, a matrix is simply a rectangular wayof storing data.

■ Matrices can have unlimited dimensions, however for ourpurposes, all matrices will be in two dimensions:

◆ Rows

◆ Columns

■ Matrices are symbolized by boldface font in text, usuallywith capital letters.

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 19 of 55

Vectors

■ A vector is a matrix where one dimension is equal to sizeone.

◆ Column vector: A column vector is a matrix of size r × 1.

◆ Row vector: A row vector is a matrix of size 1 × c.

■ Vectors allow for geometric representations of matrices.

■ The Pearson correlation coefficient is a function of the anglebetween vectors.

■ Much of the statistical theory underlying linear models(ANOVA-type) can be conceptualized by projections ofvectors (think of the dependent variable Y as a columnvector).

■ Vectors are typically symbolized by boldface font in text,usually with lowercase letters.

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 20 of 55

Scalars

■ A scalar is a matrix of size 1 × 1.

■ Scalars can be thought of as any single value.

■ The difficult concept to get used to is seeing a number as amatrix:

2.759]

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 21 of 55

Matrix Elements

■ A matrix is composed of a set of elements, each denoted it’srow and column position within the matrix.

■ For a matrix A of size r × c, each element is denoted by:

◆ The first subscript is the index for the rows: i = 1, . . . , r.◆ The second subscript is the index for the columns:

j = 1, . . . , c.

a11 a12 . . . a1c

a21 a22 . . . a1c

......

ar1 ar2 . . . arc

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 22 of 55

Transpose

■ The transpose of a matrix is simply the switching of theindices for rows and columns.

■ An element aij in the original matrix (in the ith row and jth

column) would be aji in the transposed matrix (in the jth rowand the ith column).

■ If the original matrix was of size i × j the transposed matrixwould be of size j × i.

A′ =

Overview

Introduction to SAS

Matrix Algebra

● Introduction

● Why?

● Definitions

● Proc IML Basics

● Matrices

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 23 of 55

Types of Matrices

■ Square Matrix: A matrix that as the same number of rowsand columns.

◆ Correlation and covariance matrices are examples ofsquare matrices.

■ Diagonal Matrix: A diagonal matrix is a square matrix withnon-zero elements down the diagonal and zero values forthe off-diagonal elements.

2.759 0 0

0 1.643 0

0 0 0.879

■ Symmetric Matrix: A symmetric matrix is a square matrixwhere aij = aji for all elements in i and j.

◆ Correlation/covariance and distance matrices areexamples of symmetric matrices.

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

● Singular Matrices

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 24 of 55

Algebraic Operations

■ As mentioned in the definition at the beginning of class,algebra is simply a set of math that defines basic operations.

◆ Identity

◆ Zero

◆ Addition

◆ Subtraction

◆ Multiplication

◆ Division

■ Matrix algebra is simply the use of these operations withmatrices.

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 25 of 55

Matrix Addition

■ Matrix addition is very much like scalar addition, the onlyconstraint is that the two matrices must be of the same size(same number of rows and columns).

■ The resulting matrix contains elements that are simply theresult of adding two scalars.

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 26 of 55

Matrix Addition

a11 a12

a21 a22

a31 a32

a41 a42

b11 b12

b21 b22

b31 b32

b41 b42

A + B =

a11 + b11 a12 + b12

a21 + b21 a22 + b22

a31 + b31 a32 + b32

a41 + b41 a42 + b42

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 27 of 55

IML Addition

proc iml;reset print;

A={10 15, 11 9, 1 -6};B={5 2, 1 0, 10 7};C=A+B;

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 28 of 55

Matrix Subtraction

■ Matrix subtraction is identical to matrix addition, with theexception that all elements of the new matrix are thesubtracted elements of the previous matrices.

■ Again, the only constraint is that the two matrices must be ofthe same size (same number of rows and columns).

■ The resulting matrix contains elements that are simply theresult of subtracting two scalars.

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 29 of 55

Matrix Subtraction

a11 a12

a21 a22

a31 a32

a41 a42

b11 b12

b21 b22

b31 b32

b41 b42

A − B =

a11 − b11 a12 − b12

a21 − b21 a22 − b22

a31 − b31 a32 − b32

a41 − b41 a42 − b42

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 30 of 55

IML Subtraction

A={10 15, 11 9, 1 -6};B={5 2, 1 0, 10 7};C=A-B;

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 31 of 55

Matrix Multiplication

■ Unlike matrix addition and subtraction, matrix multiplicationis much more complicated.

■ Matrix multiplication results in a new matrix that can be ofdiffering size from either of the two original matrices.

■ Matrix multiplication is defined only for matrices where thenumber of columns of the first matrix is equal to the numberof rows of the second matrix.

■ The resulting matrix as the same number of rows as the firstmatrix, and the same number of columns as the secondmatrix.

A B = C(r × c) (c × k) (r × k)

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 32 of 55

a11 a12

a21 a22

a31 a32

a41 a42

b11 b12 b13

b21 b22 b23

a11b11 + a12b21 a11b12 + a12b22 a11b13 + a12b23

a21b11 + a22b21 a21b12 + a22b22 a21b13 + a22b23

a31b11 + a32b21 a31b12 + a32b22 a31b13 + a32b23

a41b11 + a42b21 a41b12 + a42b22 a41b13 + a42b23

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 32 of 55

a11 a12

a21 a22

a31 a32

a41 a42

b11 b12 b13

b21 b22 b23

a11b11 + a12b21 a11b12 + a12b22 a11b13 + a12b23

a21b11 + a22b21 a21b12 + a22b22 a21b13 + a22b23

a31b11 + a32b21 a31b12 + a32b22 a31b13 + a32b23

a41b11 + a42b21 a41b12 + a42b22 a41b13 + a42b23

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 32 of 55

a11 a12

a21 a22

a31 a32

a41 a42

b11 b12 b13

b21 b22 b23

a11b11 + a12b21 a11b12 + a12b22 a11b13 + a12b23

a21b11 + a22b21 a21b12 + a22b22 a21b13 + a22b23

a31b11 + a32b21 a31b12 + a32b22 a31b13 + a32b23

a41b11 + a42b21 a41b12 + a42b22 a41b13 + a42b23

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 33 of 55

IML Multiplication

A={10 15, 11 9, 1 -6};B={5 2, 1 0, 10 7};C=A*T(B);D=T(B)*(A);

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 34 of 55

Multiplication and Summation

■ Because of the additive nature induced by matrixmultiplication, many statistical formulas that use:

can be expressed by matrix notation.

■ For instance, consider a single variable Xi, with i = 1, . . . , Nobservations.

■ Putting the set of observations into the column vector X, ofsize N × 1, we can show that:

X2 = X′ X

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 35 of 55

Matrix Multiplication by Scalar

■ Recall that a scalar is simply a matrix of size (1 × 1).■ Matrix multiplication by a scalar causes all elements of the

matrix to be multiplied by the scalar.■ The resulting matrix has all elements multiplied by the scalar.

a11 a12

a21 a22

a31 a32

a41 a42

s × A =

s × a11 s × a12

s × a21 s × a22

s × a31 s × a32

s × a41 s × a42

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 36 of 55

Identity Matrix

■ The identity matrix is defined as a matrix that whenmultiplied with another matrix produces that original matrix:

A I = A

I A = A

■ The identity matrix is simply a square matrix that has alloff-diagonal elements equal to zero, and all diagonalelements equal to one.

I(3×3) =

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 37 of 55

Zero Matrix

■ The zero matrix is defined as a matrix that when multipliedwith another matrix produces the matrix:

A 0 = 0

0 A = 0

■ The zero matrix is simply a square matrix that has allelements equal to zero.

0(3×3) =

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 38 of 55

Matrix “Division”: The Inverse

■ Recall from basic math that:

ba = b−1a

■ And that:a

■ Matrix inverses are just like division in basic math.

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 39 of 55

The Inverse

■ For a square matrix, an inverse matrix is simply the matrixthat when pre-multiplied with another matrix produces theidentity matrix:

A−1A = I

■ Matrix inverse calculation is complicated and unnecessarysince computers are much more efficient at finding inversesof matrices.

■ One point of emphasis: just like in regular division, divisionby zero is undefined.

■ By analogy - not all matrices can be inverted.

Overview

Introduction to SAS

Matrix Algebra

Algebra

● Algebra

● Addition

● Subtraction

● Multiplication

● Identity

● Zero

● “Division”

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 40 of 55

Singular Matrices

■ A matrix that cannot be inverted is called a singular matrix.

■ In statistics, common causes of singular matrices are foundby linear dependence among the rows or columns of asquare matrix.

■ Linear dependence can be cause by combinations ofvariables, or by variables with extreme correlations (eithernear 1.00 or -1.00).

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

● Linear Combinations

● Linear Dependencies

● Vector Length

● Inner Product

● Angle Between Vectors

● Vector Projections

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 41 of 55

Linear Combinations

■ Vectors can be combined by adding multiples:

y = a1x1 + a2x2 + . . . + akxk

■ The resulting vector, y, is called a linear combination.

■ All for k vectors, the set of all possible linear combinations iscalled their span.

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

● Vector Length

● Inner Product

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 42 of 55

Linear Dependencies

■ A set of vectors are said to be linearly dependent ifa1, a2, . . . , ak exist, and:

◆ a1x1 + a2x2 + . . . + akxk = 0.

◆ a1, a2, . . . , ak are not all zero.

■ Such linear dependencies occur when a linear combinationis added to the vector set.

■ Matrices comprised of a set of linearly dependent vectorsare singular.

■ A set of linearly independent vectors forms what is called abasis for the vector space.

■ Any vector in the vector space can then be expressed as alinear combination of the basis vectors.

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

● Vector Length

● Inner Product

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 43 of 55

Vector Length

■ The length of a vector emanating from the origin is given bythe Pythagorean formula:

Lx =√

x21 + x2

2 + . . . + x2k

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

● Vector Length

● Inner Product

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 44 of 55

Inner Product

■ The inner (or dot) product of two vectors x and y is the sumof element-by-element multiplication:

x′y = x1y1 + x2y2 + . . . + xkyk

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

● Vector Length

● Inner Product

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 45 of 55

Vector Angle

■ The angle formed between two vectors x and y is

cos(θ) =x′y√

x′x√

■ If x′y = 0, vectors x and y are perpendicular, as noted byx⊥y

■ All basis vectors are perpendicular.

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

● Vector Length

● Inner Product

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 46 of 55

Vector Projections

■ The projection of a vector x onto a vector y is given by:

x′yL2

■ Through such projections, a set of linear independentvectors can be created from any set of vectors.

■ One process used to create such vectors is through theGram-Schmidt Process.

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 47 of 55

Matrix Properties

The following are some algebraic properties of matrices:

■ (A + B) + C = A + (B + C) - Associative

■ A + B = B + A - Commutative

■ c(A + B) = cA + cB - Distributive

■ (c + d)A = cA + dA

■ (A + B)′ = A′ + B′

■ (cd)A = c(dA)

■ (cA)′ = cA′

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

Lecture #2 - 8/31/2005 Slide 48 of 55

Matrix Properties

The following are more algebraic properties of matrices:

■ c(AB) = (cA)B

■ A(BC) = (AB)C

■ A(B + C) = AB + AC

■ (B + C)A = BA + CA

■ (AB)′ = B′A′

■ For xj such that Ax j is defined:

Ax j = An

(Ax j)(Ax j)′ = A

xjx′

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

● Advanced Topics

● Determinants

● Orthogonality

● Eigenspaces

● Decompositions

Wrapping Up

Lecture #2 - 8/31/2005 Slide 49 of 55

Advanced Matrix Functions/Operations

■ We end our matrix discussion with some advanced topics.

■ All of these topics are related to multivariate analyses.

■ None of these will seem too straight forward today, but nextweek we will use some of these results to demonstrateproperties of sample statistics.

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

● Advanced Topics

● Determinants

● Orthogonality

● Eigenspaces

● Decompositions

Wrapping Up

Lecture #2 - 8/31/2005 Slide 50 of 55

Matrix Determinants

■ A square matrix can be characterized by a scalar valuecalled a determinant.

det A = |A|■ Much like the matrix inverse, calculation of the determinant is

very complicated and tedious, and is best left to computers.

■ What can be learned from determinants is if a matrix issingular.

■ Matrices with determinants that are greater than zero aresaid to be “positive definite,” a byproduct of which is that apositive matrix is non-singular.

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

● Advanced Topics

● Determinants

● Orthogonality

● Eigenspaces

● Decompositions

Wrapping Up

Lecture #2 - 8/31/2005 Slide 51 of 55

Matrix Orthogonality

■ A square matrix (A) is said to be orthogonal if:

AA ′ = A′A = I

■ Orthogonal matrices are characterized by two properties:

1. The product of all row vector multiples is the zero matrix(perpendicular vectors).

2. For each row vector, the sum of all elements is one.

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

● Advanced Topics

● Determinants

● Orthogonality

● Eigenspaces

● Decompositions

Wrapping Up

Lecture #2 - 8/31/2005 Slide 52 of 55

Eigenvalues and Eigenvectors

■ A square matrix can be decomposed into a set ofeigenvalues and eigenvectors.

Ax = λx

■ From a statistical standpoint:◆ Principal components are comprised of linear combination

of a set of variables weighed by the eigenvectors.

◆ The eigenvalues represent the proportion of varianceaccounted for by specific principal components.

◆ Each principal component is orthogonal to the next,producing a set of uncorrelated variables that may beused for regression purposes.

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

● Advanced Topics

● Determinants

● Orthogonality

● Eigenspaces

● Decompositions

Wrapping Up

Lecture #2 - 8/31/2005 Slide 53 of 55

Spectral Decompositions

■ Imagine that a matrix A is of size k × k.

■ A then has:

◆ k eigenvalues: λi, i = 1, . . . , k.

◆ k eigenvectors: ei, i = 1, . . . , k (each of size k × 1.

■ A can be expressed by:

λieie′

■ This expression is called the Spectral Decomposition, whereA is decomposed into k parts.

■ One can find A−1 by taking 1λ

in the spectral decomposition.

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

● Final Thought

● Next Class

Lecture #2 - 8/31/2005 Slide 54 of 55

Final Thought

■ The SAS learning curve issmall, but once you havethe basics, all you need isthe manual and you can doalmost anything.

■ Proc IML is useful formatrices, but probably notso useful to you.

■ Matrix algebra makes the technical things in life easier.

■ The applications of matrices will be demonstrated throughoutthe rest of this course.

Overview

Introduction to SAS

Matrix Algebra

Algebra

Vector Definitions

Matrix Properties

Advanced Topics

Wrapping Up

● Final Thought

● Next Class

Lecture #2 - 8/31/2005 Slide 55 of 55

Next Time

■ Applications of matrix algebra in statistics (Chapter 2 sectionsix and on).

■ Geometric implications of multivariate descriptive statistics(more applications).

Introduction to SAS Vectors and Matrices · PDF fileIntroduction to SAS. Vectors and Matrices...

Documents