Applied and Numerical Harmonic Analysis

Series Editor

John J. BenedettoUniversity of Maryland

Editorial Advisory Board

Akram Aldroubi Douglas CochranVanderbilt University Arizona State University

Ingrid Daubechies Hans G. FeichtingerPrinceton University University of Vienna

Christopher Heil Murat KuntGeorgia Institute of Technology Swiss Federal Institute of Technology, Lausanne

James McClellan Wim SweldensGeorgia Institute of Technology Lucent Technologies, Bell Laboratories

Michael Unser Martin VetterliSwiss Federal Institute Swiss Federal Instituteof Technology, Lausanne of Technology, Lausanne

M. Victor WickerhauserWashington University

Myoung AnAndrzej K. BrodzikRichard Tolimieri

Ideal Sequence Designin Time-FrequencySpaceApplications to Radar, Sonar,and Communication Systems

BirkhäuserBoston • Basel • Berlin

Myoung An Andrzej K. BrodzikdeciBel Research, Inc. The MITRE Corporation1525 Perimeter Parkway, Suite 500 202 Burlington Road, MS E050Huntsville, AL 35806, USA Bedford, MA 01730, USAmyoung@dbresearch.net abrodzik@mitre.org

Richard TolimieriPsypher, Inc.1327 Monte Sano Blvd. SEHuntsville, AL 35801, USArichard@psypher.org

www.birkhauser.com

Library of Congress Control Number: 2008932706

Mathematics Subject Classification (2000): 42-04, 94A99, 54A99

All rights reserved. This work may not be translated or copied in whole or in part without the writtenpermission of the publisher (Birkhäuser Boston, c/o Springer Science+Business Media, LLC, 233 SpringStreet, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarlyanalysis. Use in connection with any form of information storage and retrieval, electronic adaptation,computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden.The use in this publication of trade names, trademarks, service marks, and similar terms, even if they arenot identified as such, is not to be taken as an expression of opinion as to whether or not they are subjectto proprietary rights.

Printed on acid-free paper

© +Business Media, LLC Birkhäuser Boston , a part of Springer Science 2009

ISBN 978-0-8176-4737-7 e-ISBN 978-0-8176-4738-4DOI 10.1007/978-0-8176-4738-4

ANHA Series Preface

The Applied and Numerical Harmonic Analysis (ANHA) book series aims to providethe engineering, mathematical, and scientific communities with significant develop-ments in harmonic analysis, ranging from abstract harmonic analysis to basic appli-cations. The title of the series reflects the importance of applications and numericalimplementation, but richness and relevance of applications and implementation de-pend fundamentally on the structure and depth of theoretical underpinnings. Thus,from our point of view, the interleaving of theory and applications and their creativesymbiotic evolution is axiomatic.

Harmonic analysis is a wellspring of ideas and applicability that has flourished,developed, and deepened over time within many disciplines and by means of creativecross-fertilization with diverse areas. The intricate and fundamental relationship be-tween harmonic analysis and fields such as signal processing, partial differentialequations (PDEs), and image processing is reflected in our state-of-the-art ANHAseries.

Our vision of modern harmonic analysis includes mathematical areas such aswavelet theory, Banach algebras, classical Fourier analysis, time-frequency analysis,and fractal geometry, as well as the diverse topics that impinge on them.

For example, wavelet theory can be considered an appropriate tool to deal withsome basic problems in digital signal processing, speech and image processing, geo-physics, pattern recognition, biomedical engineering, and turbulence. These areasimplement the latest technology from sampling methods on surfaces to fast algo-rithms and computer vision methods. The underlying mathematics of wavelet theorydepends not only on classical Fourier analysis, but also on ideas from abstract har-monic analysis, including von Neumann algebras and the affine group. This leadsto a study of the Heisenberg group and its relationship to Gabor systems, and of themetaplectic group for a meaningful interaction of signal decomposition methods. Theunifying influence of wavelet theory in the aforementioned topics illustrates the jus-tification for providing a means for centralizing and disseminating information fromthe broader, but still focused, area of harmonic analysis. This will be a key role of

vi ANHA Series Preface

ANHA. We intend to publish with the scope and interaction that such a host of issuesdemands.

Along with our commitment to publish mathematically significant works at thefrontiers of harmonic analysis, we have a comparably strong commitment to publishmajor advances in the following applicable topics in which harmonic analysis playsa substantial role:

Antenna theory Prediction theoryBiomedical signal processing Radar applications

Digital signal processing Sampling theoryFast algorithms Spectral estimation

Gabor theory and applications Speech processingImage processing Time-frequency and

Numerical partial differential equations time-scale analysisWavelet theory

The above point of view for the ANHA book series is inspired by the history ofFourier analysis itself, whose tentacles reach into so many fields.

In the last two centuries Fourier analysis has had a major impact on the devel-opment of mathematics, on the understanding of many engineering and scientificphenomena, and on the solution of some of the most important problems in mathe-matics and the sciences. Historically, Fourier series were developed in the analysisof some of the classical PDEs of mathematical physics; these series were used tosolve such equations. In order to understand Fourier series and the kinds of solutionsthey could represent, some of the most basic notions of analysis were defined, e.g.,the concept of “function." Since the coefficients of Fourier series are integrals, it isno surprise that Riemann integrals were conceived to deal with uniqueness proper-ties of trigonometric series. Cantor’s set theory was also developed because of suchuniqueness questions.

A basic problem in Fourier analysis is to show how complicated phenomena,such as sound waves, can be described in terms of elementary harmonics. There aretwo aspects of this problem: first, to find, or even define properly, the harmonics orspectrum of a given phenomenon, e.g., the spectroscopy problem in optics; second,to determine which phenomena can be constructed from given classes of harmonics,as done, for example, by the mechanical synthesizers in tidal analysis.

Fourier analysis is also the natural setting for many other problems in engineering,mathematics, and the sciences. For example, Wiener’s Tauberian theorem in Fourieranalysis not only characterizes the behavior of the prime numbers, but also providesthe proper notion of spectrum for phenomena such as white light; this latter processleads to the Fourier analysis associated with correlation functions in filtering andprediction problems, and these problems, in turn, deal naturally with Hardy spaces inthe theory of complex variables.

Nowadays, some of the theory of PDEs has given way to the study of Fourierintegral operators. Problems in antenna theory are studied in terms of unimodulartrigonometric polynomials. Applications of Fourier analysis abound in signal pro-cessing, whether with the fast Fourier transform (FFT), or filter design, or the adap-

ANHA Series Preface vii

tive modeling inherent in time-frequency-scale methods such as wavelet theory. Thecoherent states of mathematical physics are translated and modulated Fourier trans-forms, and these are used, in conjunction with the uncertainty principle, for dealingwith signal reconstruction in communications theory. We are back to the raison d’etreof the ANHA series!

John J. BenedettoSeries Editor

University of MarylandCollege Park

Preface

The topic of this book is the design of sequences with good correlation properties.The ideas presented in the work evolved, in part, out of a radar project whose goal wasto develop algorithms for processing linear frequency modulated (FM) chirp pulsereflections from multiple targets and dielectric materials. An especially convenientframework for this task is given by one of the time-frequency spaces, the Zak space.The Zak space provides a natural setting for studying chirps, as they are intrinsicallytime-frequency signals and have simple realizations as modulated algebraic lines onthe Zak transform lattice. Because the replacement of time-domain analysis withtwo-dimensional Zak space analysis leads to the replacement of one-dimensionalreflections with two-dimensional images, image processing techniques can be usedto estimate target parameters, including dielectric material properties.

Complex target discrimination often requires the use of multi-beam imagingschemes. Multi-beam imaging can also be applied to aid in multi-material identi-fication and in noise, multi-path and atmospheric interference rejection. To realizethis advantage, the information contained in individual components of the echo mustbe unambiguously retrieved at the receiver. At the same time, to maintain an accept-able receiver signal-to-noise ratio, a sequence set must make maximum use of theavailable bandwidth. An accommodation of these two constraints necessitates iden-tification of a sequence set that is sufficiently large and whose individual membersinterfere with each other as little as possible.

We address these signal design problems by Zak space methods. Zak space meth-ods are used first to prove several well-known correlation properties of sampled linearFM chirp pulses. Among the theoretical results, we show that the Zak space represen-tation of a discrete linear FM chirp is the matrix product of several copies of a per-mutation matrix and a diagonal matrix. This factorization decouples the permutationmatrix from the modulating diagonal matrix. This result is the key to understandingthe central role played by the Zak transform in our approach to sequence design. Itpermits the permutation group to take a central role in the design. Because certainclasses of chirps are identical with certain communications sequences, obtained by adirect, time-domain approach, several results between the two intersect. However, theapproach described in this work is broader in that it offers a novel way of analyzing

x Preface

chirps by both algebraic and geometric means and leads to the construction of muchmore general signal sets.

Along with mathematical developments illustrated by numerous examples, thetext contains many tables. These tables compare the number and size of signal setssatisfying good correlation properties of specified periodicity attainable by the meth-ods of communication theory with those attainable by Zak space methods. While thetheoretical benefit of the Zak space approach is demonstrated, the efficacy of the ap-proach largely depends on how, and to what advantage, the newly designed sequencescan be used in applications. To address this question, several chapters describe ape-riodic correlation properties and methods for limiting bandwidth and smoothing ofanalog envelopes. In the last chapter, a list of open problems is given and directionsfor further research are discussed.

We owe a good deal to Dr. Albanese of Brooks City-Base whose patience andkeen insights provided the motivation for many of the results and, more importantly,the emphasis in the text. Through this collaboration, the mathematics of chirps andsequence design are never far removed from significant applications to radar imageprocessing and materials identification. We also thank Dr. J. Oeschger of the NavalSurface Warfare Center who introduced us to interferometric synthetic aperture sonarimage processing and requirements on sequence sets. The design of real-valued se-quence sets in Chapter 15 is mainly due to this collaboration. Lastly, we are grateful toDr. Rushanan and the late Dr. Fante, both of The MITRE Corporation, for discussionsof multiple access communication systems and multi-beam radar applications.

Myong AnAndrzej K. Brodzik

Richard Tolimieri

April 2008

Contents

ANHA Series Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Review of Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1 Ring of Integers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Vectors and Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 Tensor Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 Permutations and Permutation Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 133.1 Shift Permutations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.2 Unit Permutations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.3 Stride Permutations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4 Finite Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254.1 CT FFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.2 FT of Zero-Padded Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

5 Convolution and Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.1 Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.2 Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385.3 Characterization of Ideal Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . 415.4 Acyclic Convolution and Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . 43

6 Discrete Chirps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516.1 Correlation Properties of Discrete Chirps . . . . . . . . . . . . . . . . . . . . . . . 546.2 Fourier Transform of Discrete Chirps . . . . . . . . . . . . . . . . . . . . . . . . . . 606.3 Gauss Sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 666.4 Computation of Gauss Sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

xii Contents

7 Zak Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697.1 ZS Representation of Shifts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717.2 ZS Representation of Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . 747.3 ZS Interchange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 807.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

8 Zak Space Correlation Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

9 Zak Space Representation of Chirps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

10 ∗-Permutations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10910.1 Fixed Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11510.2 Permutation Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

11 Permutation Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11911.1 Computation in Zak Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12111.2 Ideal Correlation of Permutation Sequences . . . . . . . . . . . . . . . . . . . . 12511.3 Relaxing the ∗-Permutation Condition . . . . . . . . . . . . . . . . . . . . . . . . . 131

12 Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13712.1 Modulation N = L2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13712.2 Modulation N = L2R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14212.3 Modulation and Ideal Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

13 Sequence Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15513.1 Ideal Correlation Roots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15513.2 Numerics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

14 Echo Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16714.1 Cyclic Shifts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16814.2 Fourier Transform of Zero-Padded Vectors . . . . . . . . . . . . . . . . . . . . . 16914.3 ZS Representation of Zero-Padded Vectors . . . . . . . . . . . . . . . . . . . . . 17314.4 Modulated Permutation Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17614.5 Global Computations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17914.6 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

15 Sequence Shaping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18515.1 Bandlimiting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18515.2 Real-Valued Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

16 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

1

Introduction

In this text new methods for studying the correlation properties of polyphase se-quences are developed. These methods are applied to the design of large collectionsof polyphase sequence pairs satisfying ideal correlation and polyphase sequence setssatisfying pairwise ideal correlation. Throughout the text a sequence is said to bepolyphase if its components have equal absolute value.

Our approach is based on the finite Zak transform, and the point of departure inthe investigation is the discrete linear frequency modulated (FM) chirp. Chirps areintrinsically time-frequency signals and have sparse and highly structured represen-tations in time-frequency spaces. The correlation properties of quadratic chirps havebeen studied in [1]. We will prove several of these results using Zak space methodsas motivation for the sequence design procedures developed in this text. Among themain theoretical results, we show that the finite Zak transform of a discrete chirp isthe product of several copies of a permutation matrix and a diagonal matrix whosenonzero entries have absolute value one. This result is the key to understanding thecentral role played by the Zak transform in our approach to sequence design. This ap-proach is based on designing sequences directly in the Zak space, with separate focusgiven to permutations and modulations. In the first part of the book, we define a newclass of sequences, the permutation sequences, by the condition that their Zak spacerepresentations are permutation matrices. The permutation sequences automaticallysatisfy ideal autocorrelation. However, a pair of permutation sequences does not nec-essarily satisfy ideal cross correlation, requiring further analysis. The cornerstone ofthis analysis is the identification of a special class of permutations, the ∗-permutations.Each ∗-permutation serves as a root for a large class of permutation sequence pairssatisfying ideal correlation. As part of this investigation, we distinguish the class ofdiscrete chirps, which up to modulation are ∗-permutation sequences. In all casesthe collection of sequence pairs determined by a root can be algebraically computed.The characterization of permutation sequence pairs satisfying ideal correlation is themain result needed to construct sequence sets of modulated permutation sequencessatisfying pairwise ideal correlation.

In the second part of the book, we bring back the diagonal matrix factor. Amodulated permutation sequence is a sequence whose Zak transform is a permutation

M. An et al., Ideal Sequence Design in Time-Frequency Space, DOI 10.1007/978-0-8176-4738-4_1, © Birkhäuser Boston, a part of Springer Science+Business Media, LLC 2009

2 1 Introduction

matrix or several copies of a permutation matrix multiplied by a diagonal matrix whosediagonal entries have absolute value one. Modulating a permutation sequence changesthe shape of the sequence but not its cyclic correlation properties. A generalizationof a permutation sequence is a sequence whose Zak transform consists of more thanone permutation matrix. This sequence does not satisfy ideal autocorrelation. In thiscase modulation is required to impose ideal autocorrelation.

The main tool for studying the correlation properties of permutation sequencesand modulated permutation sequences is the Zak space correlation formula. Thisformula describes the Zak space representation of the correlation of two sequencesin terms of componentwise products of the columns of the Zak transform of the twosequences. Underlying the correlation formula is the relationship between sequenceshifts and the Zak transform of sequence shifts. This formula, together with the Zaktransform factorization result, guides the construction of new sequences.

Although we believe the Zak space sequence design approach presented here isoriginal, we are aware of and indebted to previous efforts. The nature of the discretechirp as a time-frequency signal was first recognized by Lerner [28]. The importanceof the Zak transform in the study of the continuous chirp, as well as its special rolein signal processing, was first pointed out by Janssen [25]. Janssen also derived themain properties of the Zak transform, including the Zak space correlation formula,and determined the Fourier transform and the Zak transform of the continuous chirp.There is also a wealth of literature in the use of other time-frequency transforms forstudying the properties of discrete FM chirps [3, 35].

Although the main motivation for the approach in the text comes from sonar andradar systems, there is a common link to several works in communications sequencedesign. The link to finite field sequence design [11, 12, 15, 20, 21, 22, 23, 27, 31] isless clear, but the concept of a ∗-permutation introduced in this text has similaritieswith that of shift sequences [18]. The important text by S.W. Golomb and G. Gong[20] contains a treatment of shift sequences and their application to the design ofsequences having good correlation properties.

The presentation of the material is self-contained, and background informationis given when needed. In particular, as the constructions rely on matrix algebra,tensor products and permutation groups, a brief review of these topics is provided inChapters 2 and 3. In Chapters 4 and 5 the main digital signal processing concepts,the finite Fourier transform and the correlation, are developed. Chapters 6 through 9introduce the discrete chirp and the finite Zak transform, and state the main resultson Zak space representations of chirps, including the Zak space correlation of chirps.Chapters 14 through 15 form the core of the book, as they develop the Zak space designframework of ideal sequences. Chapter 10 characterizes the set of ∗-permutations thatare associated with ideal permutation sequences. Chapter 11 analyzes properties ofpermutation sequences based on ∗-permutations and identifies several families of idealsequences. Chapter 12 investigates properties of modulation and derives the conditionfor arbitrary sequences to satisfy ideal correlation. In Chapter 13 we use results onpermutation sequence pairs to develop design strategies for constructing collections ofpermutation sequences satisfying pairwise ideal correlations. Some of these strategieslead to explicit construction, but perhaps the most interesting result is numerical

1 Introduction 3

procedures for constructing large numbers of such collections. The numerics forseveral sizes of sequence collections are given in tables. Chapters 14 and 15 addressseveral engineering issues pertinent to radar and sonar signal processing. Chapter 16outlines several outstanding time-frequency sequence design problems.

2

Review of Algebra

In this chapter we introduce some of the notation and algebra used in this text. Refer-ences include [24], [40]. Other results will be discussed as needed. Throughout thiswork N > 1 is an integer.

2.1 Ring of Integers

Z/N is the set{0, 1, . . . , N − 1}

under addition and multiplication modulo N . Z/N is a commutative ring called thering of integers modulo N . 0 is the identity for addition and 1 is the identity formultiplication. For an arbitrary integer r,

r mod N

is the unique integer in Z/N equal to r modulo N .We write

(r, N) = 1

to mean r is relatively prime to N . UN is the set of integers in Z/N that are invertibleunder the multiplication in Z/N . UN is a group under multiplication modulo N calledthe group of units of Z/N ,

UN = {r ∈ Z/N : (r, N) = 1}.

In the next chapter we identify the group of units, UN , of Z/N with a specialsubgroup of N × N permutation matrices, the group of unit permutation matrices.The unit permutation matrices play an essential part in the Zak space representationof discrete chirps in Chapter 9.

M. An et al., Ideal Sequence Design in Time-Frequency Space, DOI 10.1007/978-0-8176-4738-4_2, © Birkhäuser Boston, a part of Springer Science+Business Media, LLC 2009

6 2 Review of Algebra

Example 2.1

U5 = {1, 2, 3, 4} .

U8 = {1, 3, 5, 7} .

U9 = {1, 2, 4, 5, 7, 8} .

U15 = {1, 2, 4, 7, 8, 11, 13, 14} .

In general the ratio|UN |N

does not grow as N grows. This ratio affects the maximum order of sequence setshaving good correlation properties of size N .

For a prime p, Z/p is a field and

Up = {1, . . . , p − 1}.

Up is a cyclic group under multiplication modulo p. This means that there existsm ∈ Up such that

Up ={mj : 0 ≤ j < p − 1

}.

m is called a generator (it is not unique) of Up. In general the set of nonzero elementsof a finite field is a cyclic group under field multiplication. If p is an odd prime andr ≥ 1 is an integer, then

Upr

is a cyclic group, having pr−1(p − 1) elements.

Upr = {mj : 0 ≤ j < pr−1(p − 1)}.

We call m a generator of Upr .

Example 2.2

U7 = {1, 2, 3, 4, 5, 6} ={3j : 0 ≤ j < 6

},

where 3j is taken modulo 7, and 36 ≡ 1 mod 7.

Example 2.3

U9 = {1, 2, 4, 5, 7, 8} ={2j : 0 ≤ j < 6

},

where 2j is taken modulo 9, and 26 ≡ 1 mod 9.

For N not a prime power UN is not a cyclic group, but by the Chinese remaindertheorem UN is the group direct product

UN = Upr11

× · · · × Uprtt

,

where pr11 , . . ., prt

t are the distinct prime power factors of N .

2.2 Vectors and Matrices 7

Example 2.4 U15 = U3 × U5.

Example 2.5 Up2 , p an odd prime, consists of the p blocks of p − 1 integers

1, 2, . . . , p − 1,

p + 1, p + 2, . . . , 2p − 1,

...

p(p − 1) + 1, p(p − 1) + 2, . . . , p2 − 1.

2.2 Vectors and Matrices

C is the complex field and C1 is the multiplicative group of complex numbers ofabsolute value 1. C

N is the complex vector space of all ordered N -tuples of complexnumbers. A vector x ∈ C

N is written

x = [xn]0≤n<N =

⎡⎢⎢⎢⎣x0x1...

xN−1

⎤⎥⎥⎥⎦ .

The inner product of two vectors x and y in CN is

〈x,y〉 =N−1∑n=0

xny∗n,

where ∗ is complex conjugation. The norm of x is

‖x‖ =√

〈x,x〉.

The set of vectors{en : 0 ≤ n < N},

where en is the vector in CN having 1 in the n-th component and 0 in all other

components is an orthonormal basis of CN

〈er, es〉 ={

1, r = s0, r = s,

0 ≤ r, s < N.

When it is not clear from context we write eNn or write en ∈ C

N to indicate theappropriate space. A vector x ∈ C

N can be uniquely written as

x =N−1∑n=0

xnen, xn ∈ C.

8 2 Review of Algebra

1 is the vector in CN all of whose components are equal to 1 and 0 is the vector

in CN all of whose components are equal to 0. When it is not clear from context, we

write 1N and 0N to indicate the appropriate space. The vector 1 can be written

1 =N−1∑n=0

en.

Componentwise multiplication of vectors plays an important role in this text. Thenotation

xy

denotes the vector in CN formed by componentwise multiplication,

xy = [xnyn]0≤n<N .

In particular,

eres ={

er, r = s0, r = s,

0 ≤ r, s < N,

and

er

(N−1∑n=0

xnen

)= erx = xrer, 0 ≤ r < N.

L and K are positive integers. A complex L × K matrix X is written

X = [xl,k]0≤l<L,0≤k<K .

We usually write a complex L × K matrix in terms of its column vectors,

[x0 · · · xK−1] , xk ∈ CL

or[X0 · · · XK−1] , Xk ∈ C

L.

For this reason we setEn = en, 0 ≤ n < N.

The trace of an N × N matrix X is

Tr X =N−1∑n=0

xn,n.

The trace is defined only for square matrices.Special matrices will be used throughout this text. We introduce them now, but

study them more completely in subsequent chapters. IN is the N ×N identity matrix.The N × N shift matrix S defined by

2.2 Vectors and Matrices 9

Sx =

⎡⎢⎢⎢⎣xN−1

x0...

xN−2

⎤⎥⎥⎥⎦and the N × N time reversal matrix R defined by

Rx =

⎡⎢⎢⎢⎣x0

xN−1...

x1

⎤⎥⎥⎥⎦are examples of permutation matrices. Permutation matrices will be studied in detailin Chapter 3. When dimension must be distinguished we write SN for S and RN forR. Several elementary results about S and R will be described at this time.

S has order N ,SN = IN ,

and N is the smallest positive integer with this property. As a result S is invertibleand

S−1 = SN−1.

The componentwise product of any two distinct columns of Sn, 0 ≤ n < N , is 0 and

Tr Sn = 0, 0 < n < N.

R has order 2,R2 = IN .

The componentwise product of any two distinct columns of R is 0 and

Tr R = 1.

R and S are related byRSR = S−1.

The N × N Fourier transform matrix F is defined by

F = [wmn]0≤m,n<N , w = e2πi 1N .

When dimension must be distinguished we write F (N) for F . The N × N Fouriertransform matrix will be studied in detail in Chapter 4. This will include the importantrelationships between S, R and F .

For x ∈ CN , D(x) is the N × N diagonal matrix defined by

D(x) =

⎡⎢⎢⎢⎣x0

x1 0. . .

0 xN−1

⎤⎥⎥⎥⎦ .

10 2 Review of Algebra

Componentwise multiplication and diagonal matrix multiplication can be inter-changed,

xy = D(x)y, x, y ∈ CN .

An important example is

D = D([wn]0≤n<N

), w = e2πi 1

N .

When dimension must be distinguished we write DN for D.D has order N ,

DN = IN ,

and N is the smallest positive integer having this property.

2.3 Tensor Products

Throughout this section N = LK, where L > 1, K > 1 and M > 1 are integers.Tensor products of vectors and matrices can be found in many branches of physics

and mathematics. They have been used in digital signal processing to model manyresults and algorithms in terms of matrix factorizations. Tensor products and stridepermutations, introduced in the next chapter, form the basic building blocks of thetensor product algebra which can be used to describe complex computations and canserve as an interactive programming tool. A complete treatment of the tensor productalgebra along with the tensor product identities and its application to algorithms forthe finite Fourier transform and convolution is contained in [40] and [41].

In this work the tensor product algebra is more than an implementation tool. It isthe algebra underlying derivations and results, replacing the summations and multipleindices usually found in digital signal processing texts.

The tensor product of vectors x ∈ CL and y ∈ C

K is the vector x ⊗ y ∈ CN

defined by

x ⊗ y =

⎡⎢⎣ x0y...

xL−1y

⎤⎥⎦ .

We can view x ⊗ y as consisting of L contiguous vector segments each in CK ,

xly, 0 ≤ l < L.

Example 2.6

⎡⎣x0x1x2

⎤⎦⊗[

y0y1

]=

⎡⎢⎢⎢⎢⎢⎢⎣x0y0x0y1x1y0x1y1x2y0x2y1

⎤⎥⎥⎥⎥⎥⎥⎦ .

2.3 Tensor Products 11

The collection of tensor products

x ⊗ y, x ∈ CL,y ∈ C

K ,

spans CN , but is not a basis.

The tensor product is associative and distributive,

(x ⊗ y) ⊗ z = x ⊗ (y ⊗ z),

(x + y) ⊗ z = x ⊗ z + y ⊗ z

andx ⊗ (y + z) = x ⊗ y + x ⊗ z, x ∈ C

L, y ∈ CK , z ∈ C

M ,

but is not commutative. We also have

(αx) ⊗ y = α(x ⊗ y) = x ⊗ (αy), α ∈ C.

These results can be derived by following the definition of the tensor product.

Example 2.7 eLr ⊗ eK

s = eNs+rK .

T = RS, where R > 1, S > 1 are integers. The tensor product of an L × Rmatrix X and a K × S matrix Y is the N × T matrix X ⊗ Y defined by

X ⊗ Y = [xl,rY ]0≤l<L,0≤r<R .

We can view X ⊗ Y as consisting of LR matrix blocks

xl,rY, 0 ≤ l < L, 0 ≤ r < R,

each a K × S matrix.

Example 2.8 I2 ⊗Y = Y ⊕Y =[

Y 00 Y

], where I2 is the 2× 2 identity matrix and

⊕ is the matrix direct sum.

In general IL⊗Y is the block diagonal matrix having L copies of Y along the diagonaland 0 elsewhere.

Example 2.9

[a bc d

]⊗ I2 =

⎡⎢⎢⎢⎢⎣a 0 b 00 a 0 b

c 0 d 00 c 0 d

⎤⎥⎥⎥⎥⎦ .

12 2 Review of Algebra

Write the L × R matrix X and the K × S matrix Y in terms of their columnvectors,

X = [x0 · · · xR−1] and Y = [y0 · · · ys−1] .

Thenxr ⊗ Y = [xr ⊗ y0 · · · xr ⊗ ys−1]

andX ⊗ Y = [x0 ⊗ Y · · · xr−1 ⊗ Y ] .

These formulas can be used to show that the tensor product of matrices is associativeand distributive, but not commutative. We also have

(X ⊗ Y )(x ⊗ y) = Xx ⊗ Y y

and(X ⊗ Y )(Z ⊗ W ) = XZ ⊗ Y W

for appropriate size matrices X, Y, Z, W and vectors x and y. As a consequence, ifX and Y are invertible,

(X ⊗ Y )−1 = X−1 ⊗ Y −1.

Example 2.10 S22L = SL ⊗ I2.

In generalSK

N = SL ⊗ IK .