Post on 21-May-2018
transcript
Potpourri on(Fast Algorithms for)Structured Matrices
Vadim Olshevsky
GSU
Potpourri on(Fast Algorithms for)Structured Matrices – p.1/80
Friends and Colleagues
Parts of this work was done jointly with
A.Shokrollahi (Bell Labs and DigitalFountain), VictorPan (CUNY), and Michael Stewart (ANU and nowGSU).
Other relevant work was done jointly with:
I.Gohberg (Tel Aviv), T.Kailath (Stanford);
Georg Heinig (Germany), Leonid Lerer (Technion),Dario Fasino (Italy.
Potpourri on(Fast Algorithms for)Structured Matrices – p.2/80
The themes
Introduction and Some Examples.
Forward stable Cauchy solver.
Backward stable Hankel solver.
Displacement structure .
Tangential interpolation of the Nevanlinna-Pick type .
List decoding of the R-S codes .
Potpourri on(Fast Algorithms for)Structured Matrices – p.3/80
Some examplesToeplitz,
� � �� ��� � �
Hankel,
� �� �� � �
Vandermonde,
� �� � � �� �
Polyn, Vand. � � � � � � �� � � �
Cauchy,
� � � ���� �� �
Pick,
� � � � � � � ���� �� � �� �
Also Bezoutians, Sylvester matrices, controllability,observability, Krylov matrices and others.Also their inverses, products, Schur complements.
Potpourri on(Fast Algorithms for)Structured Matrices – p.4/80
Some examplesToeplitz,
� � �� ��� � �
Hankel,
� � �� � �
�����������������
� � � � � � � !� �
� � � � � � � ...... . . . . . . . . . ...... . . . � � � � �� !� � � � � �"
#�#�#�#�#�#�#�#$Vandermonde,
� �� � � �� �Polyn, Vand.
� � � � � � �� � � �
Cauchy,
� � � �� �� �� �Pick,
� � � � � � � ���� �� � �� �
Also Bezoutians, Sylvester matrices, controllability,observability, Krylov matrices and others.Also their inverses, products, Schur complements.
Potpourri on(Fast Algorithms for)Structured Matrices – p.4/80
Some examplesToeplitz,
� � �� ��� � �
Hankel,
� � �� � �
����������������%�
� � & !� � � & . . . ! & . . . . . . ......
!� � ! & !� ' !� � ! & !� ' & !� &
Vandermonde,
� �� � � �� �Polyn, Vand.
� � � � � � �� � � �
Cauchy,
� � � ���� �� �Pick,
� � � � � � � ���� �� � �� �
Also Bezoutians, Sylvester matrices, controllability,observability, Krylov matrices and others.Also their inverses, products, Schur complements.
Potpourri on(Fast Algorithms for)Structured Matrices – p.4/80
Some examplesToeplitz,
� � �� ��� � �
Hankel,
� �� �� � �
Vandermonde,
� �� � � �� �
Polyn, Vand. � � � � � � �� � � �
�����������
( � � � &� � !� ��
......
......
......
......( � ! � & ! � !� �!
"#�#�#�#�#$
Cauchy,
� � � ���� �� �Pick,
� � � � � � � ���� �� � �� �
Also Bezoutians, Sylvester matrices, controllability,observability, Krylov matrices and others.Also their inverses, products, Schur complements.
Potpourri on(Fast Algorithms for)Structured Matrices – p.4/80
Some examplesToeplitz,
� � �� ��� � �
Hankel,
� � �� � �
Vandermonde,
� �� � � �� �
Polyn, Vand. � � � � � � �� � � �
����������������%�
� � �� � � � � �� � � � !� � �� � �
......
......
......� � �� ! � � � �� ! � � !� � �� ! �
Cauchy,
� � � ���� �� �Pick,
� � � � � �� ���� �� � �� �
Also Bezoutians, Sylvester matrices, controllability,observability, Krylov matrices and others.Also their inverses, products, Schur complements.
Potpourri on(Fast Algorithms for)Structured Matrices – p.4/80
Some examplesToeplitz,
� � �� ��� � �
Hankel,
� �� �� � �
Vandermonde,
� �� � � �� �
Polyn, Vand. � � � � � � �� � � �
Cauchy,
� � � ���� �� �
Pick,
� � � � � � � ���� �� � �� �
�����
��)� � ) ��)� �+*
......��* � � ) ��* � �*
"#�#$
Also Bezoutians, Sylvester matrices, controllability,observability, Krylov matrices and others.Also their inverses, products, Schur complements.
Potpourri on(Fast Algorithms for)Structured Matrices – p.4/80
Some examplesToeplitz,
� � �� ��� � �
Hankel,
� �� �� � �
Vandermonde,
� �� � � �� �
Polyn, Vand. � � � � � � �� � � �
Cauchy,
� � � ���� �� �
Pick,
� � � � � � � ���� �� � �� �
�������
� � �), �� )� )� � � ) � � �), ��*� )� � �*
......� � �* , �� )�+* � � � ) � � �* , ��*� * � � �*
"#�#�#$
Also Bezoutians, Sylvester matrices, controllability,observability, Krylov matrices and others.Also their inverses, products, Schur complements.
Potpourri on(Fast Algorithms for)Structured Matrices – p.4/80
Some examplesToeplitz,
� � �� ��� � �
Hankel,
� �� �� � �
Vandermonde,
� �� � � �� �
Polyn, Vand. � � � � � � �� � � �
Cauchy,
� � � ���� �� �
Pick,
� � � � � � � ���� �� � �� �
Also Bezoutians, Sylvester matrices, controllability,observability, Krylov matrices and others.Also their inverses, products, Schur complements.
Potpourri on(Fast Algorithms for)Structured Matrices – p.4/80
Some examplesToeplitz,
� � �� ��� � �
Hankel,
� �� �� � �
Vandermonde,
� �� � � �� �
Polyn, Vand. � � � � � � �� � � �
Cauchy,
� � � ���� �� �
Pick,
� � � � � � � ���� �� � �� �
Also Bezoutians, Sylvester matrices, controllability,observability, Krylov matrices and others.Also their inverses, products, Schur complements.
Potpourri on(Fast Algorithms for)Structured Matrices – p.4/80
Recent interest
Conferences
Publishing projects
Software project
Potpourri on(Fast Algorithms for)Structured Matrices – p.5/80
Recent interest
ConferencesSanta Barbara-1996, Cortona-97, Boulder-1999, Cortona-2000, SouthHadley-2001, ???? Banff-2003 ???.
Publishing projects
Software project
Potpourri on(Fast Algorithms for)Structured Matrices – p.5/80
Recent interest
ConferencesSanta Barbara-1996, Cortona-97, Boulder-1999, Cortona-2000, SouthHadley-2001, ???? Banff-2003 ???.
Publishing projects
Edited by Kailath & Sayed, SIAM, 1999.
Edited by Bini, Tyrtyshnikov and Yalamov, Nova Publications, 2000.
Two-Volume set edited by V.O. to AMS series Contemporary Mathematics,November 2001.
2002. A special issue of LAA edited by Dewilde, Sayed & V.O., March 2002.
2002.An AMS/SIAM volume on Fast Algorithms edited by V.O.
2002. A special issue of LAA edited by D.Bini, G.Heinig and E.Tyrtyshnikov.
Software project
Potpourri on(Fast Algorithms for)Structured Matrices – p.5/80
Recent interest
ConferencesSanta Barbara-1996, Cortona-97, Boulder-1999, Cortona-2000, SouthHadley-2001, ???? Banff-2003 ???.
Publishing projects
Edited by Kailath & Sayed, SIAM, 1999.
Edited by Bini, Tyrtyshnikov and Yalamov, Nova Publications, 2000.
Two-Volume set edited by V.O. to AMS series Contemporary Mathematics,November 2001.
2002. A special issue of LAA edited by Dewilde, Sayed & V.O., March 2002.
2002.An AMS/SIAM volume on Fast Algorithms edited by V.O.
2002. A special issue of LAA edited by D.Bini, G.Heinig and E.Tyrtyshnikov.
Software project
SLAP (SLAPACK). ????
Potpourri on(Fast Algorithms for)Structured Matrices – p.5/80
Why? Full research circle.Applications. .
Mathematics..
Scientific Computing.
Potpourri on(Fast Algorithms for)Structured Matrices – p.6/80
Why? Full research circle.Applications. .
Estimation, Signal processing, Filtering, Circuit simulations, Systems andControl, Image Processing, Error-correcting codes.
Mathematics..
Scientific Computing.
Potpourri on(Fast Algorithms for)Structured Matrices – p.6/80
Why? Full research circle.Applications. .
Estimation, Signal processing, Filtering, Circuit simulations, Systems andControl, Image Processing, Error-correcting codes.
Mathematics..Various interpolation and Approximation problems, Orthogonal polynomials,Moment problems, RKHS, Commutant-lifting theory, etc.
Scientific Computing.
Potpourri on(Fast Algorithms for)Structured Matrices – p.6/80
Why? Full research circle.Applications. .
Estimation, Signal processing, Filtering, Circuit simulations, Systems andControl, Image Processing, Error-correcting codes.
Mathematics..Various interpolation and Approximation problems, Orthogonal polynomials,Moment problems, RKHS, Commutant-lifting theory, etc.
Scientific Computing.Software design. Fast Algorithms for structured matrices. A problem: speedvs. accuracy.
Potpourri on(Fast Algorithms for)Structured Matrices – p.6/80
Why? Full research circle.Applications. .
Estimation, Signal processing, Filtering, Circuit simulations, Systems andControl, Image Processing, Error-correcting codes.
Mathematics..Various interpolation and Approximation problems, Orthogonal polynomials,Moment problems, RKHS, Commutant-lifting theory, etc.
Scientific Computing.Software design. Fast Algorithms for structured matrices. A problem: speedvs. accuracy.
A RECENT DISCOVERY: Though fast algorithms were believed to betypically inaccurate, it turns out that there are better modified algorithms that BLENDSPEED AND ACCURACY. Examples.
Potpourri on(Fast Algorithms for)Structured Matrices – p.6/80
Linear predictionIn 1940 N.Wiener won a $2,350 NDRC project on “a design of a predictionapparatus in which, when one member follows the track of an airplane, anotheranticipates where the airplane is to be after a fixed lapse of time.”
If a stochastic process
-.0/ 12/ 34 2 is stationary in the wide sense, theleast-square criterion to find the prediction,
. * 5 )as
.76 � 8 ). )�:9 9 9 � 8 * . *;
leads to the Yule-Walker linear equations:<=>=>=>=>=?=@=?=?=@=?=?AB6 B � ) , , , , , , B �*B ) B6 B � ) ...
.... . .
. . .. . .
......
. . . B6 B � )B * , , , , , , B ) B6CD>D>D>D>D?D@D?D?D@D?D?E<=>=>=>=?=@=?=?=@=FA�8 )8G
...8 *CD>D>D>D?D@D?D?D@DFE
H<=>=>=>=?=@=?=?=@=FAI��
... �CD>D>D>D?D@D?D?D@DFE
9(1)
In the 1949 Wiener monograph there is an Appendix by N.Levinson with a
J K ! GL
algorithm to solve (1). Faster than general
J K ! ML .Fast algorithm
Back to Full Circle. . Potpourri on(Fast Algorithms for)Structured Matrices – p.7/80
Caratheodory-Toeplitz problemGiven: ! � �
complex numbers
- B/ 1
Find:
N K �L H B6 � B ) � � , , ,� B * � * � J KO �O * 5 )L
, analytic on the unit diskP
and positivereal:
Q 8 N K �L R �
in
P
.
Solution: the classical Schur 1917 algorithm:
Matrix interpretation: Computing the Cholesky factorization
S H T , T � for
S H<=?=?=@=?=>=>=>=>=>=?=FAB6 � B �6 B � ) , , , , , , B �*B ) B6 � B �6 B � ) ...
.... . .
. . .. . .
......
. . . B6 � B �6 B � )B+* , , , , , , B ) B6 � B �6CD?D?D@D?D>D>D>D>D>D?DFE
(
S R �
is the solvability condition).
The Cholesky factorization requires
J K ! ML flops. The Schur algorithm usesonly
J K ! GL flops. Fast algorithm.
Back to Full Circle.
Potpourri on(Fast Algorithms for)Structured Matrices – p.8/80
Speed vs accuracySpeed:
Standard algorithms Gaussian Elimination
J K ! ML flops
Fast algorithms The Levinson and the Schur
J K ! GLSuperfast algorithms The Brent-Gustafson-Yan
J K ! UWV X G !LAccuracy? Some algorithms are fast AND accurate. One example: FFT. However,too many fast algorithms are fast ways of computing garbage.
Good news. In the last few years a number of fast and accurate algorithms havebeen designed.
Bojanczyk, Brent,de Hoog, and D. R. Sweet (1995). The Schur algorithm forp.d. Toeplitz matrices is backward stable.
Gohberg, Kailath and O. (1995). A reliable algorithm for indefiniteToeplitz(+Hankel)-like matrices.
Also: I.Koltracht, S.Chandrasekaran, M.Gu, P.Van Dooren, M.Stewart, G.Heinig,M.Van Barel, L.Reichel,
Back to Full Circle. Back to ContentsPotpourri on(Fast Algorithms for)Structured Matrices – p.9/80
Floating point arithmetic
Y[Z H -\ ; ] 1 Y[Z H �9 ^ ) ^G ^ M ^`_ ^0a ^0b ^`c ^0de fg hi , � � jIf
Ylk H Nm - k 1 is a floating point representation of k thenO k � YkOO kO n oe fg h
unit roundoff
p � �4 c
in single precision.
Conclusion: We cannot expect more than 7 - 8 correct digits in the solutionscomputed in single precision.
Potpourri on(Fast Algorithms for)Structured Matrices – p.10/80
Forward error.We used several fast
J K ! G L algorithms (from Boros-Kailath-O., LAA 1999) to solve a
linear system
q k H ^
with the Hilbert coefficient matrix
q H r )�5 � 4 ) s .
Potpourri on(Fast Algorithms for)Structured Matrices – p.11/80
Forward error.We used several fast
J K ! G L algorithms (from Boros-Kailath-O., LAA 1999) to solve a
linear system
q k H ^
with the Hilbert coefficient matrix
q H r )�5 � 4 ) s .
K q � t qL K k � t kL H ^ u vWwx t kk ny K qL t qq , K7z{ | 8 } ~ � !� L ;
Potpourri on(Fast Algorithms for)Structured Matrices – p.11/80
Forward error.We used several fast
J K ! G L algorithms (from Boros-Kailath-O., LAA 1999) to solve a
linear system
q k H ^
with the Hilbert coefficient matrix
q H r )�5 � 4 ) s .n 5 10 15 20 25y 2 K SL
5e+05 2e+13 2e+17 8e+17 1e+19
0 5 10 15 2010−8
10−6
10−4
10−2
100
102
104
SIZE
RE
LATI
VE
ER
RO
R
BKO+mon
BKO+PPP
GECP
Potpourri on(Fast Algorithms for)Structured Matrices – p.11/80
Remarkable forward error boundsTHM. (Boros-Kailath-O., LAA 1999). Let
Yk be a solution tor )�� 4 �� s k H N9
computed via an algorithm from Boros-Kailath-O, LAA, 1999. If
�* �9 9 9 � � ) � �) �9 9 9 � �* ;(1)
and the RHS
N
is sign-oscillating then we have :O k � YkOO kO n� ! o � � K oG L remarkable componentwise FORWARD accuracy
Why?
Gohberg-Koltracht (1993) The structured condition numbers of Cauchy andVandermonde matrices are much less than their condition numbers.
Relevant and similar examples of accurate structure-exploiting algorithms: (i) FFT;(ii) the Björck-Pereyra algorithm (1970); (iii) Inversion of Cauchy matrices.
Back to ContentsPotpourri on(Fast Algorithms for)Structured Matrices – p.12/80
How bad are Hankel matrices
� H<=>=>=?=@=?=?=@=?=>=?A~6 ~ ) ~G , , , ~ * 4 )~ ) ~G . .
. ...~G . .. ~G * 4 M
... . .. ~G * 4 M ~G * 4 G )~ * 4 ) , , , ~G * 4 M ~G * 4 G ~G * 4 )
CD>D>D?D@D?D?D@D?D>D?ER �
Conditioning
Tyrtyshnikov (1994):
� & � � � � � & � � � � & � � !� �
No forward accuracy should be expected. What aboutBackward accuracy?
Potpourri on(Fast Algorithms for)Structured Matrices – p.13/80
AlgorithmsSlow: Gaussian elimination. Computes
YT p T
in
� H T T �Cost:
J K ! ML .
Stability:
� � � YT YT � � G � J K oL � � � G . Here o is the machine precision,p � �4 )b
.
Potpourri on(Fast Algorithms for)Structured Matrices – p.14/80
AlgorithmsSlow: Gaussian elimination. Computes
YT p T
in
� H T T �Cost:
J K ! ML .
Stability:
� � � YT YT � � G � J K oL � � � G . Here o is the machine precision,p � �4 )b
.
Potpourri on(Fast Algorithms for)Structured Matrices – p.14/80
AlgorithmsSlow: Gaussian elimination. Computes
YT p T
in
� H T T �Cost:
J K ! ML .
Stability:
� � � YT YT � � G � J K oL � � � G . Here o is the machine precision,p � �4 )b
.
Fast: (There are many...)
Cost:
J K ! GL or
J K ! UWV XG !L .
Stability: ? .
Potpourri on(Fast Algorithms for)Structured Matrices – p.14/80
AlgorithmsSlow: Gaussian elimination. Computes
YT p T
in
� H T T �Cost:
J K ! ML .
Stability:
� � � YT YT � � G � J K oL � � � G . Here o is the machine precision,p � �4 )b
.
Fast: (There are many...)
Cost:
J K ! GL or
J K ! UWV XG !L .
Stability: ? .
Fast AND Accurate. O.-Michael Stewart-2001:
Cost:
J K ! GL .
Stability:
� � � YT YT � � G � J K oL � � � G .
Potpourri on(Fast Algorithms for)Structured Matrices – p.14/80
AlgorithmsSlow: Gaussian elimination. Computes
YT p T
in
� H T T �Cost:
J K ! ML .
Stability:
� � � YT YT � � G � J K oL � � � G . Here o is the machine precision,p � �4 )b
.
Fast: (There are many...)
Cost:
J K ! GL or
J K ! UWV XG !L .
Stability: ? .
Fast AND Accurate. O.-Michael Stewart-2001:
Cost:
J K ! GL .
Stability:
� � � YT YT � � G � J K oL � � � G .
HOW ?Potpourri on(Fast Algorithms for)Structured Matrices – p.14/80
Displacement structureKailath, Kung, Morf (1979), Morf (1974), Sakhnovich (1976), Heinig-Rost (1984), Dym(RKHS), Ball-Gohberg-Rodman (interpolation), etc.
Toeplitz- like
� Displacement rank of
�g h e f
rank
K� � � �� L � � !Hankel- like
�
rank
K� � � �� �L � � !
Vandermonde- like
�
rank
K� 4 )� � � �� �L � � !
Cauchy- like
�
rank
K � � � � �� �L � � !
(Below we introduce the more general Confluent Cauchy-like matrices.)
Applications to tangential interpolation
� � H �* K �L H<=>=?=@=?=>=>=>=?=@=FA�� �
� � . . ....
. . .. . .
. . .� , , , � � �CD>D?D@D?D>D>D>D?D@DFE
; � � H ���� X K �L9
Potpourri on(Fast Algorithms for)Structured Matrices – p.15/80
Why the names?The displacement rank of a Cauchy matrix
S H r )�� 4 �� s
is equal to 1:
� � S � S� � H r ���� 4 �� s � r ��� � 4 �� s H r �� 4 ���� 4 �� s H<=>=>=>=?=@=FA� � , , , �� � , , , �
......
...� � , , , �CD>D>D>D?D@DFE
If the displ. rank
K �L � � ! then
�
is called Cauchy-like.
The Pick matrices
� H � )4 �� � ���� 5 � �� �are Cauchy-like (displ. rank H &
):
� � � � �� �� H � � � N �N �� � H<=?=@=?=>=>=FA� � N )� � NG
......� � N*
CD?D@D?D>D>DFE<A � � , , , �N � ) N �G , , , N �*CE
e f g h
Short generators
9
Potpourri on(Fast Algorithms for)Structured Matrices – p.16/80
The uniform way to capture the structureIf the displ. rank of
�
is � then one can factor (nonuniquely)
q¡ � � � q£¢ H ¤ , S He f g h¥
, ;
Thus,
�
is defined by only
& � ! entries of its generator - ¤; S 1
.
Hankel generator:
� � � �� � H:<=@=?=>=>=FA� � ~6 � ~ ) � ~G~6 ~ ) 0~G
CD@D?D>D>DFE H<=@=?=>=>=FA~6 �� ~6� ~ )� ~GCD@D?D>D>DFE
<A � � �� �CE <A ~6 � � �� ~6 ~ ) ~GCE H
Unfortunately the Hankel structure is destroyed during elementarycomputations.
Potpourri on(Fast Algorithms for)Structured Matrices – p.17/80
Composite matricesComposite matrices: Inverses of Hankel matrices,
�4 )
, their products
� ) �G ,Schur complements
�G G � � )G �4 )) ) �G )are NOT Hankel matrices themselves.
If
- � �1 are “structured”, we would like the same to be true for thecomposite matrices.
“Theorem”: The composite matrices will also be structured, if “structure” is takenas DISPLACEMENT STRUCTURE.
Displ. rank
K �L n &
Displ. rank
K �4 )L n &
.
Displ. rank
� ) �G n ¦
.
Displ. rank
K �G G � �G ) �4 )) ) � )G L n &.
The displacement approach to a problem is to solve it
in terms of generator matrices
- ¤; S 1;
and not in terms of the entries of
�
.
Potpourri on(Fast Algorithms for)Structured Matrices – p.18/80
Why it is fast?Gaussian elimination Fast Displacement�G H � § )¨G G � ) ©Wªm ) « ) ¤G H ¬¤ ) � )©Wªm )^ )
H � H �
' K ! � �L G � � K ! � �L
Cost:
�¯® & �
.
What about accuracy?
Potpourri on(Fast Algorithms for)Structured Matrices – p.19/80
How to insure stability?
� � � �� � H q � q �; ° vWw± w � H <A � � �� �CE
Generator
q
is non-unique:q³² � q´ ; where
´ �´ � H �9GOOD
´
:
Scale
q² � q <A µ �� ) ©CE so that the new columns of
q
have the same
norm.
Put
q² � q <A B � zz BCEe fg h¶· ¸ ¹ ¶º ¶ *» ¼
in a proper form
<=>=>=?=@=?=FA� �� �
......� �
CD>D>D?D@D?DFE
This algorithm is backward stable:
� � � YT YT � � n J K oL � � �
. O.-Stewart-2001.
Potpourri on(Fast Algorithms for)Structured Matrices – p.20/80
Superfast displacement algorithms
Confluent Cauchy-like matrices.
LU factorization. Solving a linear system.
Matrix-vector product.
Potpourri on(Fast Algorithms for)Structured Matrices – p.21/80
Confluent Cauchy-like matrices
Definition:
confluent Cauchy-like rank
K q½ � � � q¾¢ L � � !Matrices
q ¢ ; q¡ are assumed to be in the Jordan form.
T-like, H-like, V-like, C-like matrices are special cases of confluent Cauchy-likematrices.
We designed a new superfast algorithm for computing
� H T¿
and for solving alinear system with Confluent Cauchy-like matrix
�
.
Potpourri on(Fast Algorithms for)Structured Matrices – p.22/80
Cost
± � x À K q¡ � � � q£¢ L � � !9- | );9 9 9 ; |�· 1
are the sizes of the Jordan blocks of
q .- } );9 9 9 ; }ÂÁ 1
are the sizes of the Jordan blocks of
q ¢ .
Unified Algorithm:Ã K !L
confl. Cauchy-like
J K ! UWV XG ! , � � � Ä ·/ 3 ) ÅÆ* UWV X * ÅÆ � Ä Á/ 3 ) ¸Æ* UWV X * ¸Æ �L
Includes as special cases:Toeplitz-like
J K ! UWV XG !L
Hankel-like
J K ! UWV XG !L
Vand.-like
J K ! UWV X M !L
Cauchy-like
J K ! UWV X M !L
An important ingredient of this unified algorithms is computing
matrix-vector product .
Potpourri on(Fast Algorithms for)Structured Matrices – p.23/80
Fast algorithms. Some references
Martin Morf’s breakthrough thesis (Stanford, 1974). First fast
J K ! GL Schur-typeand Levinson-type algorithms for Toeplitz-like matrices.
Monograph by Heinig and Rost (1984). First fast
J K ! GL Levinson-type algorithms
for Vandermonde- like and Cauchy- like matrices.
Survey by Kailath and Sayed (1995). FastJ K ! GL Schur-type algorithms for basic
classes with displacement structure.
Survey by V.O. (1997) Numerical accuracy issues. Fast
J K ! GL algorithms for
confluent Cauchy-like matrices in the context of tangential interpolation .
Potpourri on(Fast Algorithms for)Structured Matrices – p.24/80
Superfast algorithms
Morf’s (1974)’s thesis. A sketch of a superfast algorithm for Toeplitz-like matrices(not clean?).
Brent-Gustafson-Yan (1979) First
J K ! UWV X G !L algorithm for solving Toeplitz (not-like ) linear equations.
Bitmead-Anderson (1980) and Morf (1980). General Toeplitz- like case,J K ! UWV XG !L .
Gohberg-O. (1994) : Cauchy-like and Vandermonde-like matrices.
Our superfast algorithmApplies to the most general confluent Cauchy-like displacement structure .
Potpourri on(Fast Algorithms for)Structured Matrices – p.25/80
MV product. Classical algorithms.
DFT matrices (i.e., Vandermonde
matrices with special nodes) Fast Fourier transform
à K !LToeplitz/Hankel matrices convolution
à K !LVandermonde matrices multipoint
polynomial evaluation
à K !L UWV X !
inverse Vandermonde matrices polynomial interpolation
à K !L UWV X !
Cauchy matrices multipoint
rational evaluation
à K !L UWV X !
inverse Cauchy matrices rational interpolation
à K !L UWV X !
à K !L H ÇÉÈÊ ! UWV X ! if the field
Ë
supports FFT’s of length !! UWV X ! UWV X UWV X ! otherwise
Potpourri on(Fast Algorithms for)Structured Matrices – p.26/80
Cost of matrix-vector product (O.-Shokrollahi (2000))
Thm. Confluent Cauchy-like matrices can be multiplied by vectors using
S K !L H J K ! UWV X ! , � � � Á�3 ) | �! UWV X !| ��
¸� 3 )
Ì � ! UWV X !Ì � �L
flops, where- | );9 9 9 ; |Í· 1
are the sizes of the Jordan blocks of
q .- } );9 9 9 ; }ÂÁ 1
are the sizes of the Jordan blocks of
q ¢ .S K !L can vary from
J K ! UWV X !L to
J K ! UWV X G !L depending upon the Jordanstructure.
Back to contents
Potpourri on(Fast Algorithms for)Structured Matrices – p.27/80
The classical scalar Nevanlinna-Pick problem
Given:points � );9 9 9 ; � * in the RHP:
Qw �/ Î �values
N );9 9 9 ; N* in the unit disk:
O N/ O � �Construct an interpolant
N K �/ L H N/ , such that
(i)
N K �L is passive O N K �L O � �in the RHP
(ii)
N K �L is rational
Potpourri on(Fast Algorithms for)Structured Matrices – p.28/80
First papers and results
Solvability condition Pick (1916):
� H � )4 �� � ���� 5 � �� � R �Cost: The Nevanlinna (1929):
J K ! GL operations.
Recall:
Standard algorithmsJ K ! ML arithm. operations
Fast algorithmsJ K ! GL
Superfast algorithms
J K ! UWV X M !L or
J K ! UWV X G !L
Is the Nevanlinna-Pick algortithm fast?
Potpourri on(Fast Algorithms for)Structured Matrices – p.29/80
The algorithm is FAST
The Nevanlinna algorithm Its matrix interpretation
computation of the interpolant
N K �L The Cholesky factorization� H T T �, [
Tis lower triangular] for the Pick
matrix
� H � )4 �� � ���� 5 � �� �
Arithmetic complexityAlgorithm Cost
The standard Cholesky
J K ! ML
The Nevanlinna algorithm
J K ! GL
We shall show how to do faster than
ÏÑÐ ÒÓ
Potpourri on(Fast Algorithms for)Structured Matrices – p.30/80
Applications to system and circuit theory
Many physical phenomena can be described by a Linear Time-Invariant system:
« K �L Ô Õ K �L Ô � K �L H Õ K �L , « K �L
Passivity
Ö O Õ K �L O n � ÖConservation of energy
Energy of input « K �L ÎEnergy of output � K �L .
Many engineering applications have been discovered, e.g., the classicalNevanlinna (1929) algorithm is known under the name “Darlington (1939)synthesis procedure”
Potpourri on(Fast Algorithms for)Structured Matrices – p.31/80
Tangential interpolationMIMO systems (multi-input-multi-output) give rise to matrix rational functions× K �L H r�Ø � � § � ¨Ù� � § � ¨ s
.
�)�G�Ú
ÔÔ
...
Ô× K �L
� )�G�Û
ÔÔ...
Ô
This gives rise to the tangential interpolation conditions:r �) , , , �Ú s , × K �/ L H r � ) , , , � Û s
Potpourri on(Fast Algorithms for)Structured Matrices – p.32/80
Confluent interpolation conditions
Given: Now for each interpolation point �/ we are given two
chains of vectors:
- �/ );9 9 9 ; �/ÝÜ ÅÆ 1 - �/ );9 9 9 ; �/ÝÜ ÅÆ 1
We consider confluent interpolation conditions:
r �/ )9 9 9 �/ÝÜ ÅÆ sßÞ<=>=>=?=@=?=@=?=?=@=?=?Aà K �/ L à á K �/ L , , , , , , â ã³ä Æå ªæ § �Æ ¨§ ÅÆ 4 )¨ç� à K �/ L . . .
...� � . . .. . .
......
. . .. . . à á K �/ L� , , , , , , � à K �/ L
CD>D>D?D@D?D@D?D?D@D?D?EH
r �/ÝÜ ) , , , �/ÝÜ ÅÆ s9
Potpourri on(Fast Algorithms for)Structured Matrices – p.33/80
Two special casesNevanlinna-Pick: �/ , à K �/ L H �/
Caratheodory-Fejer, i.e., rational passive Hermite-type interpolation:
r � � 9 9 9 � sèÞ<=>=?=@=?=?=@=?=>=>=>=?=@=?=FAà K �/ L à á K �/ L , , , , , , )§ ÅÆ 4 )¨ç à § ÅÆ 4 )¨ K �/ L
� à K �/ L . . ....� � . . .
. . ....
.... . .
. . . à á K �/ L� , , , , , , � à K �/ L
CD>D?D@D?D?D@D?D>D>D>D?D@D?DFEH
r �/ÝÜ ) , , , �/ÝÜ ÅÆ s9
or in the other words
à K �/ L H �/ ); à á K �/ L H �/ G; �& é à á á K �/ L H �/ M;9 9 9
Potpourri on(Fast Algorithms for)Structured Matrices – p.34/80
Solutions and fast
ê
algorithmsScalar case : see, e.g., the monographs by Akhiezer (1965), Krein-Nudelman
(1977) and Rosenblum and Rovnyak (1985).
Fedchina (1972): Tangential problem without confluencies (via AAK).
Kovalishina and Potapov (1974) - direct
J K ! G L algorithm for tangential problem(without confluences). Via Schur-type alg. of Potapov (1966).
Fedchina (1975): First
J K ! GL algorithm for Confluent tangential problem.
Many deep results have been obtained using different approaches:
The band-extension-method by H.Dym, I.Gohberg.
The Buerling-Lax-theorem approach by J.Ball, J.Helton.
The commutant-lifting-theorem approach by C.Foias, A.Frazho.
The reproducing-kernel-Hilbert-spaces approach by H.Dym.
The state-space approach by Van Dooren, J.Ball, I.Gohberg, L.Rodman.
The discrete-transmission-lines approach by Lev Ari, T.Kailath.
The diagonal-calculus approach by P.Dewilde.
Nudelman, Delsarte-Genin-Kamp, Sarason, Sz-Nagy-Foias, Sakhnovich,Rosenblum-Rovniak, Kimura, Limebeer-Anderson, Katsnelson, Kirstein, Fritzsche,Dijksma, Langer, Sayed-Kailath-Lev Ari-Constantinescu, Alpay, Bolotnikov
Potpourri on(Fast Algorithms for)Structured Matrices – p.35/80
Computational cost of our problem
Given:
- �/ 1 ·/ 3 ) are given ë points- |/ 1 ·/ 3 ) are their multiplicities- �/ );9 9 9 ; �/ÝÜ ÅÆ 1 ·/ 3 ) are the associated chains- �/ );9 9 9 ; �/ÝÜ ÅÆ 1 ·/ 3 )
The cost
S K !L is (O.-Shokrollahi (1999)):
S K !L H J K Ã K !L , UWV X ! , � � � Ä ·/ 3 ) ÅÆ* UWV X * ÅÆ �L
where ! H | )�9 9 9 � |�·
à K !L HÇíìîì[Èìlì[Ê! UWV X ! if the field
Ë
supports FFT’s
of length !! UWV X ! UWV X UWV X ! otherwise
Potpourri on(Fast Algorithms for)Structured Matrices – p.36/80
Two simplest cases (with FFT)
One point. Tangential Caratheodory-Fejer. If ë H �
(so ! H | )) then
S K !L H J K ! UWV XG !Ln points. Tangential Nevanlinna-Pick. If ë H ! then
S K !L H J K ! UWV X M !L
Compares favorably with the earlier
J K ! GL bounds.
Back to contentsPotpourri on(Fast Algorithms for)Structured Matrices – p.37/80
CodesCodes are used when messages are to be transmitted over noisy channels.
E S S A GM E S S A GM EA
E S S A GM E
E S S A GM E
E S S A GM E
E S S A GM E
E S S A GM E
S S A GM EA
With coding
Without coding
Potpourri on(Fast Algorithms for)Structured Matrices – p.38/80
Redundnancy. Linear codes
/g h e fï ï ï ï ï ï ï ï ïe f g h*
A
Ì
-dimensional subspace of
ð* Ùis called an
� !; Ì � Ù-code.Its elements are called codewords.
The minimum distance is the the minimum distance between two distinctcodewords.
Minimum distance H minimum distance to the zero
Potpourri on(Fast Algorithms for)Structured Matrices – p.39/80
Error correctionA code of minimum distance
µ
is capable of correcting up to 8ñ H K µ � �Lò &errors.
Hamming balls ofradius e
e
d
Potpourri on(Fast Algorithms for)Structured Matrices – p.40/80
Two basic problems in coding
Design a good code, i.e., with a large minimum distance.
Design fast decoding algorithms.
Potpourri on(Fast Algorithms for)Structured Matrices – p.41/80
Reed-Solomon CodesGiven: �);9 9 9 ; �* distinct elements of
ð Ù9 .
Encode
- k );9 9 9 ; k/ 1îó ô õ N K �)L ;9 9 9 ; N K �* L öwhere
Ì n !
where
N K �L H k )� kG ��:9 9 9 � k/ �/ 4 );
Potpourri on(Fast Algorithms for)Structured Matrices – p.42/80
Excellent minimal distance of the RS Codes
The above code has dimension
Ì
.
What about minimum distance? (distance to zero)Theorem. Nonzero polynomial of degree
K Ì � �L
over a field can have at most| K Ì � �L
roots in the field.Implication:
the above code has minimum distance ! � K Ì � �L.
This is excellent: Increasing ! by
&
increases the minimum distance by
�
.
What about decoding algorithms?
Potpourri on(Fast Algorithms for)Structured Matrices – p.43/80
Encoding
N K �/ L H k )� kG �/ � k M �G/ �9 9 9 � k/ �/ 4 )
<=@=?=?=@=?=?A� �) �G ) , , , �/ 4 ))� �G �G÷G , , , �/ 4 )G
......
......� �* �G * , , , �/ 4 )*
CD@D?D?D@D?D?E<=>=>=?=@=?=>=>=>=FAk6 k )kG
...k/ 4 )CD>D>D?D@D?D>D>D>DFE
H<=@=?=?=@=?=?AN K �)LN K �G L
...N K �* LCD@D?D?D@D?D?E9
Decoding?
Potpourri on(Fast Algorithms for)Structured Matrices – p.44/80
Welch-Berlekamp Method
Let
K � );9 9 9 ; �* L have distance
n K ! � ÌLò &
from codeword
õ N K �)L ;9 9 9 ; N K �* L ö;Nø ð Ù�. �úù/ .Find � ø ð Ù�. �£ù § * 5/ ¨û G and
~ø ð Ù�. �£ù § * 4/ ¨û G such that
ü � H �;9 9 9 ; !ñ ~ K � �L � �� � K ��L H �9 Then
N K. L H � K. L~ K. L9
vanishes at these pointsh
sent word
received word
f
Error locator polynomial
Potpourri on(Fast Algorithms for)Structured Matrices – p.45/80
Matrix interpretation
! H �
and
Ì H ¦
:
<=?=?=@=?=>=>=>=>=>=?=@=?=FA� �) �G ) �M ) �_ ) � ) � ) �)� �G �GýG �MG �_G �G �G �G� �M �G M �MýM �_ M � M � M �M� �_ �G _ �M_ �_ý_ �_ �_ �_� �a �G a �Ma �_ a �a �a �a� �b �G b �Mb �_ýb �b �b �bCD?D?D@D?D>D>D>D>D>D?D@D?DFE
e f g hDispl. rank 2
<=?=@=?=?=@=?=>=>=>=>=>=?=@=?=>=>=FA�6 � )� G� M� _ ~6 ~G
CD?D@D?D?D@D?D>D>D>D>D>D?D@D?D>D>DFEH �9
Potpourri on(Fast Algorithms for)Structured Matrices – p.46/80
RS codes via displacement
<=>=>=FA)� ª
. . . )�þCD>D>DFE<=>=>=FA� �) �G ) �M ) �_ ) � ) � ) �)
......
...� �b �G b �Mb �_ýb �b �b �bCD>D>DFE �
<=?=@=FA� �) �G ) �M ) �_ ) � ) � ) �)
......
...� �b �G b �Mb �_ÿb �b �b �bCD?D@DFE
<=@=?=>=>=>=?=@=?=>=>=>=?=?A� � � � � � �� � � � � � �� � � � � � �� � � � � � �� � � � � � �� � � � � � � �� � � � � � � �CD@D?D>D>D>D?D@D?D>D>D>D?D?E
H
<=>=?=FA*
� � � � �
*
�
......
......
...
*
� � � � �
*
�CD>D?DFE
e f g h
rank 2
Potpourri on(Fast Algorithms for)Structured Matrices – p.47/80
List-Decoding of RS Codes
What if the number of errors is larger than
K ! � ÌLò &
?
Potpourri on(Fast Algorithms for)Structured Matrices – p.48/80
Bivariate InterpolationGiven: Points
K �); � )L ;9 9 9 ; K �* ; �Í* L over a field
Ë
, and integers
� n µ6 n µ ) n , , , n µ�� .Want: Nontrivial Polynomial
� K �; �L ñ H Ä � �36 ~ �K �L � �
with
� w X K ~ �L n µ �, such that� K � �; � �L H �
for
� H �;9 9 9 ; !.Existence?
N
exists, if
Ä � �36 K µ �� �L R !.�ñ H ~6 6 � ~6 ) �� ~6 G �G � ~ )6 � � ~ ) ) �� � ~G6 �G9We have
����������������
� �) �G ) � ) � ) �) �G )� �G �G÷G �G �G �G �GG� �M �G M � M � M �M �G M� �_ �G _ �_ �_ �_ �G_� �a �G a �a �a �a �Ga�
��������
���������������������~6 6 ~6 )~6 G~ )6 ~ ) )~G6�
����������H �9
Potpourri on(Fast Algorithms for)Structured Matrices – p.49/80
List decoding via displacement (O.-Shokrollahi)<=>=?=@=?=FA� �)
. . . � �aCD>D?D@D?DFE
<=>=>=?=@=?=@=FA� �) �G ) � ) � ) �) �G )� �G �G÷G �G �G �G �GG� �M �G M � M � M �M �G M� �_ �G _ �_ �_ �_ �G_� �a �G a �a �a �a �GaCD>D>D?D@D?D@DFE
�<=?=@=?=>=>=>=FA� �) �G ) � ) � ) �) �G )� �G �GýG �G �G �G �GG� �M �G M � M � M �M �G M� �_ �G _ �_ �_ �_ �G_� �a �G a �a �a �a �GaCD?D@D?D>D>D>DFE
<=?=@=?=?=@=?=>=>=>=FA� � � � � �� � � � � �� � � � � �� � � � � �� � � � � �� � � � � �CD?D@D?D?D@D?D>D>D>DFE
H<=>=>=?=@=?=?=>A�ò �) � )ò �)� �G ) �G )ò )� � ) �)�ò �G �Gò �G � �GÿG �GGò G � �G �G�ò �M � Mò �M � �G M �G Mò M � � M �M�ò �_ �_ò �_ � �G _ �G_ò _ � �_ �_�ò �a �aò �a � �G a �Gaò a � �a �aCD>D>D?D@D?D?D>E
<=>=FA� � � � � �� � � � � �� � � � � �CD>DFE9
Potpourri on(Fast Algorithms for)Structured Matrices – p.50/80
Appendix 1.Stability test for matrix polynomials
The classical Schur-Cohn test needs
J K ! G L to check discrete-time stability (rootsin the unit circle) of a polynomial
m K �L with
� w X m K �L H !.Matrix polynomals? T K �L H � * � � � * 4 ) q * 4 )�9 9 9 � � q )� q6 ;
where
q/ are |Þ | matrices.
The stability of
T K �L can be checked in
J K ! m{ � G !L operations, provided | issmall.
Potpourri on(Fast Algorithms for)Structured Matrices – p.51/80
Appendix 2.Schur-type algorithm for Hessenberg displacementstructure
Let
rank
K q¢ � � � q � L � � !In [K99] it is claimed: “The restriction that
- q ¢ ; q¡ 1are lower triangular is the most
general one that allows recursive triangular factorization (cf. a result in [LK86]).”
Heinig-O. In fact,
�
can be recursively factored as� H T¿
also in cases- q¢ ; q½ 1
are non-triangular, e.g., Hessenberg. Cost: ! G operations.
Potpourri on(Fast Algorithms for)Structured Matrices – p.52/80
Appendix 3.How bad are Pick matrices?
Bad news: The cond.number of real p.d. Pick matrices
�growths exponentially
with its size !.Good news: Pick matrices � H r ©� 5 ©��� 5 � � scan be effectively preconditioned by the Cauchy matrices
S H r )�� 5 � � s
generated by the same nodes.
Potpourri on(Fast Algorithms for)Structured Matrices – p.53/80
Appendix 3.Esponential ill-conditionig
THM:
Pick matrices are BAD: If
� H r ©� 5 ©��� 5 � � s R �theny G K �L R ë * 4 )
.
Not very bad, though: ë H � � ª 5 � �� ª4 ��� � G R �.
Potpourri on(Fast Algorithms for)Structured Matrices – p.54/80
Appendix 3.A numerical illustration. Cauchy matrices
0 5 10 15 20100
102
104
106
108
n
(d)
0 5 10 15 20100
102
104
106
108 (a)
n0 5 10 15 20
1
2
3
4
5
6
n
(b)
0 5 10 15 20100
101
n
(c)
Figure 0: (a)y G K �* L , (b)y G K �* Lò y G K �* 4 )L
, (c)
Ky G K �* L L )û *
,(c) The optimal nodes. Potpourri on(Fast Algorithms for)Structured Matrices – p.55/80
Appendix 3.Gram matrices
Let we have a certain inner product
� N K �L ; � K �L �
, and a space of functionsspanned by
- ë )K �L ; ëG K �L ;9 9 9 ; ë * K �L 1
. Then
� */ 3 ) k/ ë/ K �L ; */ 3 ) ^/ ë/ K �L � H r ^ )9 9 9 ^ * s Ã
<=@=?=FAk )
...k*CD@D?DFE;
where
à H r � ë/ K �L ; ë � K �L � sis called the Gram matrix (moment matrix).
Hankel matrices are the Gram matrices for ë/ K �L H �/
and� N K �L ; � K �L � H � �» N K �L � K �L � K �L µ �, where � K �L
is a certain generating function.
Toeplitz matrices are the Gram matrices for ë/ K �L H �/
and� N K �L ; � K �L � H � G ¢6 N K 8 ��L � K 8 ��L � K 8 ��L µ�
, where � K �L
is a certain generatingfunction.
Potpourri on(Fast Algorithms for)Structured Matrices – p.56/80
Appendix 3.Generating function for Pick matrices
Gram matrices: If
� N; � ��� H �» 4 » N K �L � K �L � K �L µ �then
� H r � )� 4 ��Æ ; )�4 �� � ��s
If
� R �
then the above � K �L can be chosen to be a good generating function, i.e.,
� � | n � K �L n Ã
on
�
.
Potpourri on(Fast Algorithms for)Structured Matrices – p.57/80
Appendix 3.Cauchy matrices. Constant generating function
If � K �L H B{ !z }
then
� H r � ¶ * Á ¸�� 5 � � s
is a Cauchy matrix.
Example: The famous Hilbert matrix
� H<=>=>=>=?=@=?=>=>=>=?=@=?=?=@=?=>=>=>=FA� )G )M )_ , , ,
)G )M )_ )a)M )_ )a )b
)_ )a )b )c)a )b )c )d
CD>D>D>D?D@D?D>D>D>D?D@D?D?D@D?D>D>D>DFE�/ H �ò & � K Ì � �L
Hilbert matrix is a standard test matrix form numer. algorithms because of itsill-conditioning. Potpourri on(Fast Algorithms for)Structured Matrices – p.58/80
Appendix 3.Preconditioning
If
� � | n � K �L n Ã
is a good generating function for
�, and
Sis a Cauchy
matrix with the same nodes as
�
, then
y G K Ë4 ) �L n Ã|9
Potpourri on(Fast Algorithms for)Structured Matrices – p.59/80
Appendix 3.Numerical illustration
5 10 151
1.2
1.4
1.6
1.8
2 (a)
n5 10 15
1
1.2
1.4
1.6
1.8
2 (b)
n
5 10 151
1.2
1.4
1.6
1.8
2 (c)
n5 10 15
1
1.2
1.4
1.6
1.8
2 (d)
n
Figure 0: Eigenvalues of preconditioned Pick matrices. � K �L H � � K �G � �L 4 )
.(a) �/ H &/ 4 )
, (b) �/ H &4/ 5 )
, (c) �/ H & G/ 4 * 4 )
, (d) Optimalnodes.
Potpourri on(Fast Algorithms for)Structured Matrices – p.60/80
Appendix 4.Vandermonde matrices. Multipoint polynomial evaluation.
Matrix-vector product:<=>=>=>=>=?=FA� �) �G ) , , , �* 4 ))� �G �G÷G , , , �* 4 )G
......
......� �* �G * , , , �* 4 )*
CD>D>D>D>D?DFE<=>=>=>=>=?=FAk6 k )
...k* 4 )CD>D>D>D>D?DFE H<=>=>=>=>=?=FAN K �)LN K �GL
...N K �* LCD>D>D>D>D?DFE
Polynomial multipoint evaluation ofN K �L H k6 � k ) �� kG �G �9 9 9 � k* 4 ) �* 4 )
at �); �G;9 9 9 ; �*9
Matrix-vector product for the inverse of Vandermonde matrix: interpolation.
Discrete Fourier transforms : a special case. �/ H 8 ��K G ¢ �/ * L
Potpourri on(Fast Algorithms for)Structured Matrices – p.61/80
Appendix 4Cauchy matrices. Multipoint rational evaluation.
Matrix-vector product:<=>=>=FA)� ª4 � ª , , , )� ª4 ��
......)��� 4 � ª , , , )��� 4 ��
CD>D>DFE<=>=>=FAk6
...k* 4 )CD>D>DFE H<=>=>=FAN K �)L
...N K �* LCD>D>DFE
Rational multipoint evaluation of
N K �L H k6 ��� � )�9 9 9 � k* 4 ) ��� �+*9
at �);9 9 9 ; �*
Matrix-vector product for the inverse of Cauchy matrix: rational interpolation.
Potpourri on(Fast Algorithms for)Structured Matrices – p.62/80
Appendix 4.Toeplitz matrices. Discrete Convolution.
Polynomial multiplication (discrete convolution):K k6 � k )� � kG � G �9 9 9 L K ^6 � ^ )� � ^G � G �9 9 9 L H
k6 ^6 � K k )^6 � k6 ^ )L � � K kG ^6 � k )^ )� k6 ^ ML � G �9 9 9
Matrix-vector product:<=?=@=?=?=@=?=>=>=?Ak6 k ) k6kG k ) k6
.... . .
. . .. . ., , , kG k ) k6
CD?D@D?D?D@D?D>D>D?E<=?=@=?=?=@=?=>=>=?A^6 ^ )^G
...
CD?D@D?D?D@D?D>D>D?EH
<=?=@=?=?=@=?=>=>=?Ak6 ^6k )^6 � k6 ^ )kG ^6 � k )^ )� k6 ^ M
...
CD?D@D?D?D@D?D>D>D?E
Potpourri on(Fast Algorithms for)Structured Matrices – p.63/80
Appendix 5.A “challenging” interpolation problem
Given:
& ! pairwise different numbers
- z );9 9 9 ; z * ; } );9 9 9 ; } * 1.
Construct: a rational scalar function
� K �L
such that� K z �L H �
and z �’s are the only zeros of
� K �L
.�4 )K } �L H �
and
} �’s are the only poles of
� K �L.N K �L H �
.
?
Potpourri on(Fast Algorithms for)Structured Matrices – p.64/80
Appendix 5.A solution via a structured matrix
� K �L H � � r � , , , � s <=>=>=FA)�4 ¸ ª
. . . )�4 ¸�CD>D>DFE<=>=>=FA)Á ª4 ¸ ª , , , )Á ª4 ¸�
......)Á � 4 ¸ ª , , , )Á � 4 ¸�
CD>D>DFE4 ) <=>=>=FA�
... �CD>D>DFE
Clearly
} �’s are the only poles of
� K �L
.
Clearly
N K �L H �
.
� K z/ L H � � r )Á Æ 4 ¸ ª , , , )Á Æ 4 ¸� s <=>=>=FA)Á ª4 ¸ ª , , , )Á ª4 ¸�
......)Á � 4 ¸ ª , , , )Á � 4 ¸�
CD>D>DFE4 ) <=>=>=FA�
... �CD>D>DFE H �9
z �’s are the only zeros of� K �L
.
Potpourri on(Fast Algorithms for)Structured Matrices – p.65/80
Appendix 5.Interpolation and Structured matrices
Remark that in the above simplest scalar rational interpolation problem� K z �L H �
.�4 )K } �L H �
.
the Cauchy matrix appears.
It turns out that varous structured matrices naturally appear in a number ofinterpolation problems.
Potpourri on(Fast Algorithms for)Structured Matrices – p.66/80
Appendix 5 .State-space formulas for
� � � �
and structured matrices!
Many interpolation problems have state space solutions of the form
� � � �
= � + S¢ � � " # $&% � � � �4 ) ¤
All the interpolation data are captured by- S¢ ; q£¢ ; q¡ ; ¤ 1
and
�
is determinedfrom the displacement equation
� q£¢ # q¡ � � ¤ S¢
Potpourri on(Fast Algorithms for)Structured Matrices – p.67/80
Appendix 5.Homogenuous tangential interpolation problem
Given:
points �);9 9 9 ; �* ø '
points � );9 9 9 ; �+* ø '
rows
^ );9 9 9 ; ^ * ø ' )( ¥
columns B );9 9 9 ; B * ø ' ¥( )
Construct a rational �Þ � matrix function
) K �L with value� ¥at infinity such that
1.The zeros of
� w u ) K �L occur only at
- �/ 1and
the zeros of
� w u )4 )K �L occur only at- �/ 1
.
2. Left int. cond.:
� w u ) K �L has a simple zero at �/ and
^/ , ) K �/ L H �
3. Right int. cond.:
� w u )4 )K �L has a simple zero at �/ and
)4 )K �/ L , B/ H �
Potpourri on(Fast Algorithms for)Structured Matrices – p.68/80
Appendix 5.Solution as given in BGR(1990)
) K �L H � ¥� S¢ K � �* � q¾¢ L 4 ) �4 ) ¤ where q¾¢ H ���� X r � )9 9 9 �* s; q½ H ���� X r �)9 9 9 �* s;
S¢ H r B ) , , , B�* s; ¤ H<=@=?=FA^ )
...^ *CD@D?DFE
and
� H r ��* � ��� 4 �� s
is obtained from
q½ � � � q¾¢ H ¤ , S¢
Back to Displacement Back to uniform
Potpourri on(Fast Algorithms for)Structured Matrices – p.69/80
Appendix 5.Two realizations
Right int.c. for
)4 )
define
- S¢ ; q£¢ 1
in
) K �L H � � S¢ K � � � q£¢ L 4 ) ¤¢(*).
Left int.c. for
)
define
- q ; ¤ 1
in
)4 )K �L H � � S K � � � q L 4 ) ¤ (**) .
Invert (*):
) K �L H � � S¢ K � � � q (¢ L 4 ) ¤¢ whereq (¢ H q¢ � ¤¢ S¢
(***).
Then (**) and (***) are similar:S¢ H S �; ¤¢ H �4 ) ¤ ; � q (¢ H q½ �
Hence
� q¢ � q½ � H � ¤¢ S¢ H ¤ S¢ and
) K �L H � � S¢ K � � � q¾¢ L 4 ) �4 ) ¤ 9
Potpourri on(Fast Algorithms for)Structured Matrices – p.70/80
Appendix 5.Symmetries
If
× K �L � � × K �L H �
on
� �
then ×4 ) H � × K �L � �and hence� � H � �� �.S¢ H � ¤ � .
× K �L H � � � ¤ � K � � � q¾¢ L 4 ) �4 ) ¤ 9
where q � � � q � H ¤ , � , ¤ �
Potpourri on(Fast Algorithms for)Structured Matrices – p.71/80
Appendix 6.Homogenuous confluent �
-contractive interpolation
Given :points
- �/ 1
inside RHP
Chains of
�Þ +
vectors
- «/ );9 9 9 ; «/ÝÜ ÅÆ 1,
Construct : a rational
+Þ +
matrix function
× K �L s.t.
(i) Tangential homogenuous confluent interpolation conditions:
r «/ )9 9 9 «/ÝÜ ÅÆ s<=>=>=?=@=?=>=>=>=>=>=?=@=?=FA× K �/ L × á K �/ L , , , , , , , ã ä Æå ªæ § �Æ ¨§ ÅÆ 4 )¨ç� × K �/ L . . .
...� � . . .. . .
......
. . .. . . × á K �/ L� , , , , , , � × K �/ L
CD>D>D?D@D?D>D>D>D>D>D?D@D?DFEH r � , , , � s9
(ii)
�-unitary on
� � × K �L , � , × � K �L H �; � ø � ��-contractive inside RHP
× K �L , � , × � K �L n �; Qw � Î �
Potpourri on(Fast Algorithms for)Structured Matrices – p.72/80
Appendix 6.BGR-90 solution and confluent Cauchy-like matrices
Solution (Ball-Gohberg-Rodman):
× K �L H � � � ¤ � K � � � q¾¢ L 4 ) �4 ) ¤ 9Solvability condition :
� R �
, where
q ¢ � � � q �¢ H ¤ � ¤ �
q¾¢ H<=>=?=@=?=@=FA� Å ª K � )L � Å- K �GL
. . . � Å�. K �·LCD>D?D@D?D@DFE�
; ¤¢ H<=?=?=@=?=>=>=>=?=@=?=>=>=>=?=@=?=@=?=?=@=?=>=>=>=?=@=?=>=>=FA�) ) � ) )
......�)Ü ÅÆ � )Ü ÅÆ�G ) �G )
......�GÜ ÅÆ �GÜ ÅÆ
......
......�· ) �· )
......�·Ü Å. �·Ü Å.
CD?D?D@D?D>D>D>D?D@D?D>D>D>D?D@D?D@D?D?D@D?D>D>D>D?D@D?D>D>DFE
Potpourri on(Fast Algorithms for)Structured Matrices – p.73/80
Appendix 6.Example: Representations of scalar rational functions
State-space form:
× K �L H � � S¢ K � � � q¾¢ L 4 ) �4 ) ¤ H
� �/10g h e fr � , , , � s
§ � 24 3 0 ¨å ªg h e f<=?=>=FA)�4 � ª
. . . )�4 ��CD?D>DFE
�4 )g h e f<=?=>=FA)� ª4 � ª , , , )� ª4 ��
......)��� 4 � ª , , , )��� 4 ��
CD?D>DFE4 ) 465g h e f<=?=>=FA�
... �CD?D>DFE
Factored form:
× K �L H �4 � ª�4 � ª , �4 � -�4 �- ,9 9 9 , �4 � ��4 ��
Complexity of evaluation of
� � � �
State-space form:
J K ! ML flops to invert
�
. 7 Factored form :
J K !L .
Potpourri on(Fast Algorithms for)Structured Matrices – p.74/80
Appendix 6.Factorization of rational matrix functions� � � � � � � � � & � � �
First-order
) )K �L .�
-contractive functions.Potapov (1960).Belevitch (1963).Youla (1964).Alpay-Dym (1986).
General functions.Dewilde-Vanderwalle (1975).Dewilde-Van Dooren (1981)Sakhnovich (1976)Bart-Gohberg-Kaashoek-Van Dooren (1979)Gohberg-Olshevsky (1994).
For a superfast divide-and-conquer algorithm we need a higher-order factorization× K �L H ) )K �L , )G K �LPotpourri on(Fast Algorithms for)Structured Matrices – p.75/80
Appendix 6.Divide-and-conquer factorization
× K �L8
)G K �L9) )K �L
8
) )G K �L9) ) )K �L
89
, , ,8 × G K �L9× )K �L
Higher-order
) )K �L (needed for superfast algorithms).
Sakhnovich (1976).
Bart-Gohberg-Kaashoek-Van Dooren (1980), Bart-Gohberg-Kaashoek(1979).
Gohberg-O. (1994). Alpay-Dym (1994), Dym (1996).
Potpourri on(Fast Algorithms for)Structured Matrices – p.76/80
Appendix 6.The first factor in
� �� � � � � � � & � � �
does not requirecomputations
Let
× K �L H � � S¢ K � � � q¾¢ L 4 ) �4 )) ¤ H
� � r S¢Ü ) S¢Ü G s K � � � <A q¢Ü ) �: q¢Ü G
CEL 4 ) <A � ) ) � )G�G ) �G GCE 4 ) <A ¤ Ü )¤ Ü GCE
Then
) )K �L H � � S¢Ü ) , K � � Ú ª� q¢Ü )L 4 ) , � ) )4 ) , ¤ Ü );
What about & � � �
?
Potpourri on(Fast Algorithms for)Structured Matrices – p.77/80
Appendix 6.Sakhnovich’s and better formulas for& � � � � " ; � K &L% � � " # $&% ; & � � � !� �& < K &L=
Sakhnovich formulas
S § G ¨¢ H K S¢Ü ) � )G � S¢Ü G �G GL �4 )G ; ¤ § G ¨ H �G K �G ) ¤ Ü )� �G G ¤ Ü G L
lead to a very slow
J K ! _L algorithm.
The formulas
S § G ¨¢ H S¢Ü G � S¢Ü ) �4 )) ) � )G ¤ § G ¨ H ¤ Ü G � �G ) �4 )) ) ¤ Ü )9
lead to a superfast D&C-algorithm.
Potpourri on(Fast Algorithms for)Structured Matrices – p.78/80
Appendix 6.Superfast D&C-factor algorithm
Input:
× )K �L H � � S § )¨¢ K � � � q£¢ L 4 ) �4 )) ¤ § )¨
Output:Minimal factorization
× )K �L H ) )K �L , )G K �L ;The inverse
×4 )) K �L H � � S § )¨ K � � � q¡ L 4 ) � ) ¤ § )¨¢ 9
Steps: 1. Write the first factor
) )K �L H � � S § )¨¢Ü ) , K � � � q¢Ü )§ )¨L 4 ) , �4 )) ) , ¤ § )¨ Ü )9
2. Call the algorithm D&C-Factor to
) )K �L to obtain its inverse
)4 )) K �L .
3. Compute the factor
)G K �L H � � S § G ¨¢ K � � � q¢Ü GL 4 ) �4 )G ¤ § G ¨ 9
4. Apply the algorithm D&C-Factor to)G K �L to obtain its inverse
)4 )G K �L .
5. Multiply the factors in
×4 )) K �L H )4 )G K �L , )4 )) K �L9
Potpourri on(Fast Algorithms for)Structured Matrices – p.79/80