CS235102 CS235102 Data StructuresData Structures

Chapter 2 Arrays and StructuresChapter 2 Arrays and Structures

2.4 The sparse matrix ADT (1/18)2.4 The sparse matrix ADT (1/18) 2.4.1 Introduction2.4.1 Introduction

In mathematics, a matrix contains In mathematics, a matrix contains mm rowsrows and and nn columnscolumns of elements, we write of elements, we write mmnn to to designate a matrix with designate a matrix with mm rows and rows and nn columns. columns.

5*36*615/15 8/36

sparse matrixdata structure?

2.4 2.4 The sparse matrix ADT (2/18)The sparse matrix ADT (2/18)

Structure 2.3 Structure 2.3 contains our contains our specification of specification of the matrix ADT.the matrix ADT. A minimal set of A minimal set of

operations operations Matrix creationMatrix creation AdditionAddition MultiplicationMultiplication TransposeTranspose

2.4 2.4 The sparse matrix ADT (3/18)The sparse matrix ADT (3/18)

The standard representation of a matrix is a two The standard representation of a matrix is a two dimensional array defined asdimensional array defined as

a[MAX_ROWS][MAX_COLS] We can locate quickly any element by writing We can locate quickly any element by writing aa[[i i ][ ][ j j ]]

Sparse matrix wastes spaceSparse matrix wastes space We must consider alternate forms of representation.We must consider alternate forms of representation. Our representation of sparse matrices should store only Our representation of sparse matrices should store only

nonzero elements.nonzero elements. Each element is characterized byEach element is characterized by <row, col, value>.

2.4 2.4 The sparse matrix ADT (4/18)The sparse matrix ADT (4/18)

We implement the We implement the CreateCreate operation as operation as below:below:

2.4 2.4 The sparse matrix ADT (5/18)The sparse matrix ADT (5/18) Figure 2.4(a) shows how the sparse matrix of Figure Figure 2.4(a) shows how the sparse matrix of Figure

2.3(b) is represented in the array 2.3(b) is represented in the array aa.. Represented by a two-dimensional array.Represented by a two-dimensional array. Each element is characterized by Each element is characterized by <row, col, value>.<row, col, value>.

transpose

row, column in ascending order

# of rows (columns) # of nonzero terms

2.4 2.4 The sparse matrix ADT (6/18)The sparse matrix ADT (6/18) 2.4.2 Transpose a Matrix2.4.2 Transpose a Matrix

For each For each rowrow i i take element <i, j, value> and store it in element <j, i, value> of take element <i, j, value> and store it in element <j, i, value> of

the transpose.the transpose. difficulty: difficulty: where to put <j, i, value>where to put <j, i, value>

((00, , 00, 15) ====> (, 15) ====> (00, , 00, 15), 15)((00, , 33, 22) ====> (, 22) ====> (33, , 00, 22), 22)((0,0, 55, -15) ====> (, -15) ====> (55, , 00, -15), -15)((11, , 11, 11) ====> (, 11) ====> (11, , 11, 11), 11)Move elements down very often.Move elements down very often.

For all elements in For all elements in columncolumn j, j, place element place element <i, j, value> in element in element <j, i, value>

2.4 2.4 The sparse matrix ADT (7/18)The sparse matrix ADT (7/18) This algorithm is incorporated in transpose This algorithm is incorporated in transpose

(Program 2.7).(Program 2.7).

Scan the array “columns” times.The array has “elements” elements.

==> O(columns*elements)

For all elements in column j

For all columns i

Assign A[i][j] to B[j][i]

place element place element <i, j, value> in element in element <j, i, value>

EX: A[6][6] transpose to B[6][6]

Set Up row & column in B[6][6]

Row Col Value

0 6 6 8

i=0 j=1

1 0 0 152 0 4 913 1 1 11

And So on…

Matrix A Row Col Value

i=0 j=1a[j]= 0 == i

i=0 j=2i=0 j=2

a[j]=3 != ii=0 j=3i=0 j=3

a[j] = 5 != ii=0 j=4i=0 j=4

a[j].col = 1 a[j].col != ii=0 j=5i=0 j=5

a[j].col = 2 a[j].col != ii=0 j=6i=0 j=6

a[j].col = 3 != ii=0 j=7i=0 j=7

a[j].col = 0 == ii=0 j=8i=0 j=8

a[j].col = 2 != ii=1 j=1i=1 j=1

a[j].col = 0 != ii=1 j=2i=1 j=2

a[j].col = 3 != ii=1 j=3i=1 j=3

a[j].col = 5 != ii=1 j=4 i=1 j=4

a[j].col = 1 == ii=1 j=5i=1 j=5

a[i].col = 2 != ii=1 j=6i=1 j=6

a[j].col = 3 != ii=1 j=7i=1 j=7

a[j] = 0 != ii=1 j=8i=1 j=8

a[i].col = 2 != i

2.4 2.4 The sparse matrix ADT (8/18)The sparse matrix ADT (8/18) Discussion:Discussion: compared with 2-D array compared with 2-D array

representationrepresentation O(columns*elements)O(columns*elements) vs. vs. O(columns*rows)O(columns*rows) elements --> columns * rows when non-sparse,elements --> columns * rows when non-sparse,

O(columnsO(columns22*rows)*rows)

Problem:Problem: Scan the array “columns” times. Scan the array “columns” times. In fact, we can transpose a matrix represented as a In fact, we can transpose a matrix represented as a

sequence of triples in O(columns + elements) time.sequence of triples in O(columns + elements) time.

Solution:Solution: First, determine the number of elements First, determine the number of elements

in each column of the original matrix. in each column of the original matrix. Second, determine the starting positions of each row Second, determine the starting positions of each row

in the transpose matrix.in the transpose matrix.

2.4 2.4 The sparse matrix ADT (9/18)The sparse matrix ADT (9/18) Compared with 2-D array representation:

O(columns+elements) vs. O(columns*rows) elements --> columns * rows O(columns*rows)

For columns

columns

For elements

For elements

Cost:Additional row_terms and starting_pos arrays are required.Let the two arrays row_terms and starting_pos be shared.

For columns

Buildup row_term& starting_pos

transpose

2.4 2.4 The sparse matrix ADT (10/18)The sparse matrix ADT (10/18) After the execution of the third After the execution of the third forfor loop, the loop, the

values of values of row_termsrow_terms and and starting_posstarting_pos are: are:

transpose

[0] [1] [2] [3] [4] [0] [1] [2] [3] [4] [5][5]

row_terms = 2 1 2 2 0 1row_terms = 2 1 2 2 0 1starting_pos = 1 3 4 6 8 8starting_pos = 1 3 4 6 8 8

Matrix A Row Col Value

[0] [1] [2] [3] [4] [5]row_terms

starting_pos

#col = 6#term = 6

0 0 0 0 0 01 1 11 1 22 2

1 3 4 6 8 8

Row Col Value

1 0 0 150 6 6 8

[0] [1] [2] [3] [4] [5]row_terms = 2 1 2 2 0 1starting_pos = 1 3 4 6 8 8

[0] [1] [2] [3] [4] [5]row_terms = 2 1 2 2 0 1starting_pos = 2 3 4 6 8 8

[0] [1] [2] [3] [4] [5]row_terms = 2 1 2 2 0 1starting_pos = 2 3 4 7 8 8

6 3 0 22

[0] [1] [2] [3] [4] [5]row_terms = 2 1 2 2 0 1starting_pos = 2 3 4 7 8 9

8 5 0 -15

[0] [1] [2] [3] [4] [5]row_terms = 2 1 2 2 0 1starting_pos = 2 4 4 7 8 9

3 1 1 11

[0] [1] [2] [3] [4] [5]row_terms = 2 1 2 2 0 1starting_pos = 2 4 5 7 8 9

4 2 1 3

7 3 2 -6

[0] [1] [2] [3] [4] [5]row_terms = 2 1 2 2 0 1starting_pos = 2 4 5 8 8 9

[0] [1] [2] [3] [4] [5]row_terms = 2 1 2 2 0 1starting_pos = 3 4 5 8 8 9

2 0 4 91

5 2 5 28

[0] [1] [2] [3] [4] [5]row_terms = 2 1 2 2 0 1starting_pos = 3 4 6 8 8 9Matrix A

Row Col Value

I = 1 I = 2 I = 3 I = 4 I = 5 I = 6 I = 7 I = 8

2.4 2.4 The sparse matrix ADT (11/18)The sparse matrix ADT (11/18) 2.4.3 Matrix multiplication2.4.3 Matrix multiplication

Definition:Definition: Given Given AA and and BB where where AA is is mmnn and and BB is is nnpp, the , the

product matrix product matrix DD has dimension has dimension mmpp. Its <. Its <ii, , jj> > element is element is

for 0 for 0 ii < < mm and 0 and 0 jj < < pp..

Example:Example:

1

0

n

kkjikij bad

2.4 2.4 The sparse matrix ADT (12/18)The sparse matrix ADT (12/18)

Sparse Matrix MultiplicationSparse Matrix Multiplication Definition: [Definition: [DD]]m*pm*p=[=[AA]]m*nm*n* [* [BB]]n*pn*p

Procedure: Fix a row of A and find all elements Procedure: Fix a row of A and find all elements in columnin column j j of of BB for for jj=0, 1, …, =0, 1, …, pp-1.-1.

Alternative 1.Alternative 1.Scan all of Scan all of BB to find all elements in to find all elements in j j..

Alternative 2.Alternative 2.Compute the transpose of Compute the transpose of BB. . (Put all column elements consecutively)(Put all column elements consecutively) Once we have located the elements of row Once we have located the elements of row ii of of AA and column and column

jj of of BB we just do a merge operation similar to that used in the we just do a merge operation similar to that used in the polynomial addition of 2.2polynomial addition of 2.2

2.4 2.4 The sparse matrix ADT (13/18)The sparse matrix ADT (13/18)

General case:General case:

dij=ai0*b0j+ai1*b1j+…+ai(n-1)*b(n-1)j

Array A is grouped by i, and after transpose, array B is also grouped by j

a a0* d b*0

b a1* e b*1

c a2* f b*2

g b*3

The multiply operation generate entries: a*d , a*e , a*f , a*g , b*d , b*e , b*f , b*g , c*d , c*e , c*f , c*g

The sparse matrix ADT (14/18)The sparse matrix ADT (14/18)

An ExampleAn ExampleA = 1 0 2 BT = 3 -1 0 B = 3 0 2 -1 4 6 0 0 0 -1 0 0 2 0 5 0 0 5

a[0] 2 3 5 bt[0] 3 3 4 b[0] 3 3 4

[1] 0 0 1 bt[1] 0 0 3 b[1] 0 0 3

[2] 0 2 2 bt[2] 0 1 -1 b[2] 0 2 2

[3] 1 0 -1 bt[3] 2 0 2 b[3] 1 0 -1

[4] 1 1 4 bt[4] 2 2 5 b[4] 2 2 5 [5] 1 2 6

row row rowcol col colvalue value value

a[0] 2 3 5 bt[0] 3 3 4 b[0] 3 3 4

[1] 0 0 1 bt[1] 0 0 3 b[1] 0 0 3

[2] 0 2 2 bt[2] 0 1 -1 b[2] 0 2 2

[3] 1 0 -1 bt[3] 2 0 2 b[3] 1 0 -1

[4] 1 1 4 bt[4] 2 2 5 b[4] 2 2 5 [5] 1 2 6

row row rowcol col colvalue value value

Totalb = 4

Totala = 5

rows_a = 2

cols_a = 3

cols_b = 3

row_begin = 1

row = 0

Totald = 0

[6] 2 [5] 3 0

Totalb = 4Totala = 5 rows_a = 2

cols_a = 3cols_b = 3

row_begin = 1row = 0

Totald = 0

column

i

Variable Value

1

a[0] 2 3 5 [1] 0 0 1 [2] 0 2 2 [3] 1 0 -1 [4] 1 1 4 [5] 1 2 6 [6] 2

bt[0] 3 3 4

bt[1] 0 0 3

bt[2] 0 1 -1

bt[3] 2 0 2

bt[4] 2 2 5

bt[5] 3 0

0j 1

row 0

sum 3

22

A[0][0]*B[0][0]

3

row_begin 1

D

d[1] 0 0 31

2

02

2

4

A[0][0]*B[0][2]

12

3

5

A[0][2]*B[2][2]

0

d[2] 0 2 12

1

6

23

3

7

3

1

And So on…

2.4 2.4 The sparse matrix ADT (15/18)The sparse matrix ADT (15/18) The programs 2.9 and 2.10 can obtain the product matrix The programs 2.9 and 2.10 can obtain the product matrix

DD which multiplies matrices which multiplies matrices AA and and BB..

a × b

2.4 2.4 The sparse matrix ADT (16/18)The sparse matrix ADT (16/18)

2.4 2.4 The sparse matrix ADT (17/18)The sparse matrix ADT (17/18) Analyzing the algorithmAnalyzing the algorithm

cols_b * termsrow1 + totalb +cols_b * termsrow1 + totalb +cols_b * termsrow2 + totalb +cols_b * termsrow2 + totalb +… +… +cols_b * termsrowp + totalbcols_b * termsrowp + totalb= cols_b * (termsrow1 + termsrow2 + … + termsrowp)+= cols_b * (termsrow1 + termsrow2 + … + termsrowp)+rows_a * totalbrows_a * totalb= cols_b * totala + row_a * totalb= cols_b * totala + row_a * totalb

O(cols_b * totala + rows_a * totalb)O(cols_b * totala + rows_a * totalb)

2.4 2.4 The sparse matrix ADT (18/18)The sparse matrix ADT (18/18) Compared with matrix multiplication using arrayCompared with matrix multiplication using array

for (i =0; i < rows_a; i++)for (i =0; i < rows_a; i++) for (j=0; j < cols_b; j++) { for (j=0; j < cols_b; j++) { sum =0; sum =0; for (k=0; k < cols_a; k++) for (k=0; k < cols_a; k++) sum += (a[i][k] *b[k][j]); sum += (a[i][k] *b[k][j]); d[i][j] =sum; d[i][j] =sum; } }

O(rows_a * cols_a * cols_b) vs. O(rows_a * cols_a * cols_b) vs. O(cols_b * total_a + rows_a * total_b)O(cols_b * total_a + rows_a * total_b)

optimal case:optimal case:total_a < rows_a * cols_a total_b < cols_a * cols_btotal_a < rows_a * cols_a total_b < cols_a * cols_b

worse case:worse case:total_a --> rows_a * cols_a, or total_a --> rows_a * cols_a, or total_b --> cols_a * cols_btotal_b --> cols_a * cols_b