6136 …islab.snu.ac.kr/upload/statistical_recovery_of_block... · 2018. 8. 13. · 6136...

6136 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 63, NO. 22, NOVEMBER 15, 2015

Statistical Recovery of SimultaneouslySparse Time-Varying Signals FromMultiple Measurement Vectors

Jun Won Choi, Member, IEEE, and Byonghyo Shim, Senior Member, IEEE

Abstract—In this paper, we propose a new sparse signal recoveryalgorithm, referred to as sparse Kalman tree search (sKTS), thatprovides a robust reconstruction of the sparse vector when thesequence of correlated observation vectors are available. Theproposed sKTS algorithm builds on expectation-maximization(EM) algorithm and consists of two main operations: 1) Kalmansmoothing to obtain the a posteriori statistics of the source signalvectors and 2) greedy tree search to estimate the support of thesignal vectors. Through numerical experiments, we demonstratethat the proposed sKTS algorithm is effective in recovering thesparse signals and performs close to the Oracle (genie-based)Kalman estimator.

Index Terms—Compressed sensing, simultaneously sparsesignal, multiple measurement vector, expectation-maximization(EM) algorithm, maximum likelihood estimation.

I. INTRODUCTION

O VER the years, there has been growing interest in the re-covery of high dimensional signals from a small number

of measurements. This new paradigm, so called compressedsensing (CS), relies on the fact that many naturally acquired highdimensional signals inherently have low dimensional structure.In fact, since many real world signals can be well approximatedas sparse signals (i.e., only a few entries of signal vector arenonzero), CS techniques have been applied to a variety of appli-cations including data compression, source localization, wire-less sensor network, medical imaging, data mining, to name justa few.Over the years, various signal recovery algorithms for CS

have been proposed. Roughly speaking, these approaches arecategorized into two classes. The first approach is based on adeterministic signal model, where an underlying signal is seen

Manuscript received November 13, 2014; revised May 07, 2015 and July20, 2015; accepted July 21, 2015. Date of publication July 31, 2015; date ofcurrent version October 07, 2015. The associate editor coordinating the reviewof this manuscript and approving it for publication was Prof. Andre Almeida.This work was supported by the National Research Foundation of Korea (NRF)grant funded by the Korean government (MSIP-2014R1A5A1011478) and theMinistry of Education (NRF-2014R1A1A2055805).J. Choi is with Department of Electrical Engineering, Hanyang University,

Seoul 133-791, Korea (e-mail: [email protected]).B. Shim is with Institute of New Media and Communications and School of

Electrical and Computer Engineering, Seoul National University, Seoul 151-742, Korea (e-mail: [email protected]).Color versions of one or more of the figures in this paper are available online

at http://ieeexplore.ieee.org.Digital Object Identifier 10.1109/TSP.2015.2463259

as a deterministic vector and the sparsity promoting cost func-tion (e.g., -norm) is employed to solve the problem. These ap-proaches include the basis pursuit (BP) [1], orthogonalmatchingpursuit (OMP) [2] and its variants [3]–[6]. The second approachis based on the probabilistic signal model, where the signal spar-sity is described by the a priori distribution of the signal andBayesian framework is employed in finding the sparse solution[7], [8].When the multiple measurement vectors (MMV) from dif-

ferent source signals with common support are available, accu-racy of the sparse signal recovery can be improved dramaticallyby performing joint processing of these vectors [9]–[15]. Sincethe algorithms based on MMV usually performs better thanthose relying on single measurement vector, many efforts havebeenmade in recent years to develop an efficient sparse recoveryalgorithm. The MMV-based recovery algorithms targeted forthe deterministic signal recovery include the mixed-norm solu-tion [9], [10] and convex relaxation [11] while the probabilisticapproaches include the MMV sparse Bayesian learning (SBL)method [12], block-SBL [13], auto-regressive SBL [14], andKalman filtering-based SBL (KSBL) [15].In this work, we are primarily concerned with the MMV-

based signal recovery problem when the observation vectors aresequentially acquired. To be specific, we express the ob-servation vector acquired at time index as

(1)

where is the system matrix, is the sourcesignal vector, and are the noise vector. We assumethat is modeled as a zero mean complex Gaussian randomvector, i.e., . Our goal in this setup is to estimate thesource signal using the sequence of the observationswhen 1) the source signal is sparse (i.e., the number nonzeroelements in is small) and 2) the dimension of the observationvector is smaller than that of the source vector . Inparticular, we focus on the scenario where the nonzero elementsof change over time with certain temporal correlations. Inthis scenario, we assume that correlated time-varying signals arewell modeled by Gauss-Markov process. Note that this model isuseful in capturing local dynamics of signals in linear estimationtheory [25].The main purpose of this paper is to propose a new statistical

sparse signal estimation algorithm for the sequential observa-tion model we just described. The underlying assumption usedin our model is that the nonzero amplitude of the sparse signalsis changing in time, leading to different signal realizations for

1053-587X © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

CHOI AND SHIM: STATISTICAL RECOVERY OF SIMULTANEOUSLY SPARSE TIME-VARYING SIGNALS FROM MULTIPLE MEASUREMENT VECTORS 6137

Fig. 1. A record of the channel impulse response of underwater acoustic chan-nels measured off the coast of Martha‘s Vinyard, MA, USA.

each measurement vector, yet the support of the signal ampli-tude is slowly varying so that the support remains unchangedover certain consecutive measurement vectors. We henceforthrefer to this model as simultaneously sparse signal with locallycommon support since the support of the sparse signal remainsconstant over the fixed interval under this assumption. Many ofsignal processing andwireless communication systems are char-acterized by this model. For example, this model matches wellwith the characteristics of multi-path fading channels for wire-less communications where the channel impulse responseshould be estimated from the received signal . Fig. 1 showsa record of the channel impulse responses (CIR) of underwateracoustic channels (represented over the propagation delay andtime domain) measured from the experiments conducted in At-lantic ocean in USA [22]. We observe that when compared tothe amplitude of the channel taps, the sparsity structure of theCIR is varying slowly. Thus, we can readily characterize thistime-varying sparse signal using the correlated random processalong with a deterministic binary parameter representing the ex-istence of the signal.In recovering the original signal vector from the mea-

surement vectors, we use the modified expectation-maximiza-tion (EM) algorithm [23]. The proposed scheme, dubbed assparse-Kalman-Tree-Search (sKTS), consists of two main op-erations: 1) Kalman smoothing to gather the a posteriori statis-tics of the source signals from individual measurement vectorwithin the block of interest and 2) identification of the supportof the sparse signal vector using a greedy tree search algorithm.Treating the problem to identify the sparsity structure of thesource signal as a combinatorial search, we propose a simple yeteffective greedy tree search algorithm that examines the smallnumber of promising candidates among all sparsity parametervectors in the tree.There exist several approaches to estimate the time-varying

sparse signals under MMV model. In [16], [17], reweightedoptimization has been modified for the sequential dynamic

filtering. In [15], modified SBL algorithm has been suggestedto adopt autoregressive modeling. In [20], EM-based adaptive

filtering scheme has been proposed in the context of sparsechannel estimation. Other than these, notable approachesinclude turbo approximate message passing (AMP) [19],Lasso-Kalman [21], and Kalman filtered CS [18]. We note thatour work is distinct from these approaches in the followingtwo aspects. First, in contrast to the previous efforts usingcontinuous (real-valued) parameters to describe signal sparsityin [15]–[17], the proposed method employs the deterministicdiscrete (binary) parameter vector that captures the on-offstructure of signal sparsity. Due to the use of deterministicparameter vector, an effort to deal with the probabilistic modelon signal sparsity is unnecessary. Also, since the search space isdiscretized, identification of parameter vector is done by the ef-ficient search algorithm. Second, while the recent work in [18]estimates signal amplitude using Kalman smoother and thenidentifies the support of sparse signal by the thresholding of theinnovation error norm, our work pursues direct estimation ofthe binary parameter vector using the modified EM algorithm.We note that a part of this paper was presented in [24]. The

distinctive contribution of the present work is that the algo-rithm is developed in a more generic system model and prac-tical issues (e.g., parameter estimation and iteration control) andreal-time implementation issues are elaborated. Further, exten-sive simulations for the practical applications are conducted todemonstrate the superiority of the proposed method.The rest of this paper is organized as follows. In Section II,

we briefly explain the sparse signal model and then presentthe proposed method. In Section III, we discuss the applica-tion of the proposed algorithm in the wireless channel estima-tion. In Section IV, the simulation results are provided, andSection V concludes the paper.Notation: Uppercase and lowercase letters written in bold-

face denote matrices and vectors, respectively. Superscriptsand denote transpose and conjugate transpose (her-

mitian operator), respectively. denotes the conjugationof the complex number . indicates an -norm of a vector.For the -norm, we abbreviate a subscript for simplicity.

is a diagonal matrix having elements only on the maindiagonal. and denote the real and imaginary partsof , respectively. denotes the expectation of a randomvariable and denotes the conditional expectationof given . means the expectation of giventhe deterministic parameter . The notations for covariancematrices are given byand . means the probability ofthe event . denotes a trace operation of the matrix .

is the element-by-element product (Hadamard product)of the matrices and . denotes the th coordinate vector.

II. PROPOSED SPARSE SIGNAL ESTIMATION TECHNIQUE

In this section, we consider the statistical estimation of thetime-varying sparse signals from the sequentially collected ob-servation vectors. As mentioned, our approach is based on theassumption that the support of the sparse signal varies slowly intime so that the multiple measurement vectors sharing commonsupport can be used to improve the estimation quality of thesparse signals. In this section, we first describe the simultane-ously sparse signal model and then present the proposed sparsesignal estimation scheme.


A. Simultaneously Sparse Signal ModelWe express a time-varying sparse signal as a product of

a vector of random processes describing the amplitudes ofnonzero entries in and the vectorindicating the existence of signal. That is,

(2)

where is the block index, the entry of is either 0 or 1 de-pending on the existence of the signal

(3)

and the time-varying amplitude is modeled as Gauss-Markov random process

(4)where is the process noise vectorand is the state update matrix. Note that the blockindex is associated with the interval of the length

. As mentioned, we assume that the support of theunderlying sparse signals is locally time-invariant so that isconstant in a block of consecutive measurement vectors. Usingthis together with the observation model in (1), we obtain thesimultaneously sparse signal model

(5)

Since follows Gaussian distribution for the given , the apriori distribution of the source signal can be described by

where

B. Derivation of Statistical Sparse Signal EstimationWhen the multiple measurement vectors

in the th block are available, the maximumlikelihood (ML) estimate of is expressed as

(6)

where and is the sparsityorder (the number of nonzero entries) of . Note that the sub-script denotes the set of time indices for the th block.Note also that the ML estimate is chosen among all candi-dates satisfying the sparsity constraint . Once

is obtained, the amplitude vectors is estimated using thelinear estimator such as the Kalman smoother. Then andthe estimate of are combined to produce a final estimate of

. Note that if the estimation of is correct, we can obtainthe best achievable estimate of , which is equivalent to thesolution attainable by so called “Oracle estimator”. Since theML problem in (6) involves the marginalization over all pos-sible combinations of the latent variables , finding out the

solution using the direct approachwould be computationally un-manageable. Perhaps, a better way to deal with the problem athand is to use the EM algorithm. Recall that the EM algorithmis an efficient means to find out the ML estimate or maximum aposteriori (MAP) estimate of statistical signal model in the pres-ence of unobserved latent variables. The EM algorithm gener-ates a sequence of estimates by alternatingtwo major steps (E-step and M-step), which are given, respec-tively• Expectation step (E-step)

(7)

• Maximization step (M-step)

(8)

where is the estimate of at the -th iteration. Althoughone cannot guarantee finding out the global optimal solution of(6) using the EM algorithm, we will empirically show thatcan be estimated accurately with a proper initialization of(see Section IV).

C. The E-StepThe goal of the E-step is to obtain a simple expression of the

cost metric using the simultaneously sparse signalmodel. First, is expressed as

(9)

(10)

Noting that and, we have

(11)

(12)

where and are the terms independent of . From (7) and(12), we further have (see Appendix A)

(13)


Let and be the conditional mean and covarianceof when and are given, i.e.,

Now we turn to the estimation of the a posteriori statisticsand . In our work, we estimate and

using Kalman smoothing [25]. When is given, from (5),the system equations for Kalman smoothing becomes

(14)

We employ the fixed-interval Kalman smoothing algo-rithm performing sequential estimation of and

via forward and backward recursions in a blockof observations . Let and be the condi-tional mean and covariance given the first observationvectors, i.e., and

, then the fixed-in-terval Kalman smoothing algorithm is summarized as• Forward recursion rule:

(15)(16)

(17)(18)

(19)

• Backward recursion rule:

(20)(21)

(22)

Using and can be rewritten as

(23)

Note that the second term in the right-hand side of (23) is ex-pressed as (see Appendix B)

(24)

From (23) and (24), we have

(25)

Further, by denoting

(26)

(27)

we have

(28)

In summary, the E-step performs the Kalman smoothing opera-tion in (15)–(22) to estimate and and also opera-tions in (26) and (27) to compute and used in the com-putation of .

D. M-Step

In the M-step, we find maximizing in (28) as

(29)

where . In finding , weneed to check all possible combinations satisfying the sparsityconstraint . Since this brute force search isprohibitive for practical values of , we consider a computa-tionally efficient search algorithm returning a sub-optimal solu-tion to the problem in (29). The proposed approach, which inessence builds on the greedy tree search algorithm, examinescandidate vectors to find out the most promising candidate ofin a cost effective manner. The tree structure used for the pro-posed greedy search algorithm is illustrated in Fig. 2. Startingfrom a root node of the tree (associated with ),we construct the layer of the tree one at each iteration. In thefirst layer of the tree, only one entry of is set to one. For ex-ample, the nodes in the first layer of the tree are expressed as

. As the layer in-creases, one additional entry is set to one and thus entries ofare set to one in the -th layer (see Fig. 2). At

each layer of the tree, we evaluate the cost functionfor each node and then choose the best nodes maximizing thecost function. The rest of nodes are discarded from the tree. Thecandidates of associated with the best nodes at each layerare called “survival list”. For each node in the survival list, weconstruct the child nodes in the second layer by setting


Fig. 2. Illustration of the proposed greedy tree search algorithm for the size of the source vector and the sparsity order . Since the number ofcandidates chosen in each layer is set to nodes with the largest values are chosen in each layer.

TABLE ISUMMARY OF THE GREEDY TREE SEARCH ALGORITHM

one additional entry of to one1. Note that since we do not dis-tinguish the order of the bit assertion in , two or more nodesmight represent the same realization of during this process(see Fig. 2). When duplicate nodes are identified, we keep onlyone and discard the rest from the tree. After removing all du-plicate nodes, we choose the best nodes and then move onto the next layer. This process is repeated until the tree reachesthe bottom layer of the tree. We note that since the tree searchcomplexity is proportional to the depth of the tree , the di-mension of source vector , and the number of nodes beingselected , one can easily show that the complexity of the pro-posed tree search is . Hence, with small values ofand , the computational complexity is reasonably small andproportional to the dimension of the source signal vector.The proposed tree search algorithm is summarized in Table I.It is worth mentioning that one important issue to be consid-

ered is how to estimate the sparsity order . One simple wayis to use the simple correlation method, where the observationvectors are correlated with the column vectors of and ischosen as the number of the column vectors whose absolute cor-relation exceeds the predefined threshold. While this approachis simple to implement, the performance might be affected bythe estimation quality of . One can alternatively consider a

simple heuristic that terminates the tree search when a big dropin the cost metric is observed.

After all iterations are finished (i.e., ) and is ob-tained, we use the Kalman smoother once again to compute

. The final estimate of is expressed as

(30)

E. Iteration ControlIn this subsection, we discuss how to configure the control

parameters in performing the iterations of the EM algorithm.In each iteration, the proposed scheme estimates the support

of the sparse signal vector under the sparsity constraint. Since the tree search to identify the support

of is based on the greedy principle, it is possible that thesupport elements might not be accurately identified, especiallyfor the initial iterations where the cost metricis not so accurate. In order to reduce the chance of missingnonzero entries of sparse vector in early iterations, we searchfor the sparse signal vector under relaxed sparsity constraintin the beginning and then gradually reduce the sparsity order

as the number of iterations increases. Let the sparsity orderparameter used for the th iteration be . Then, we usesufficiently large value of initially2 and then decreases

monotonically (i.e., ) until equals thetarget sparsity order . In doing so, we can substantially reducethe chance of missing support elements and at the same timegradually improve the estimation quality of . The summaryof the proposed algorithm is presented in Table II.

F. Estimation ofIn many practical scenarios, and in the dynamic

sparse model (see (5)) are unknown and hence should be

1For example, if is in the survival list, then the childnodes of becomes .

2Initially, we set (i.e., ) since we have noknowledge on the sparsity structure of the source signal vector in the beginning.


TABLE IISUMMARY OF THE PROPOSED ALGORITHM

estimated. We are often interested in the scenario where the el-ements of the signal vector are uncorrelated with each otherand and are fixed over the block of interval . Then,and have a diagonal form, i.e.,and . Typically,

can be determined using the temporal correlations of the thelement of , which can be known a priori or estimated fromthe data separately. While the estimation of is relatively easy,such is not the case for . One accurate but computationallyexpensive approach is to estimate these variances using EMformulation. In this work, we do not pursue this approach due tocomplexity concern and compute a rough estimate of from

, where is the th column ofthe matrix . After running all iterations, the refined estimateof can be obtained by taking sample covariance matrix ofthe estimated signal vectors in the processing block. Then, theestimate of is used for the final Kalman smoothing stepdescribed in Section II.C.

G. Real-Time ImplementationSince the proposed algorithm we described in the previous

subsections performs batch processing by running several itera-tions of E-step andM-step in a block, it might not be suitable forreal-time applications. By slightly modifying the algorithm, onecan reduce the latency and also speed up the operations substan-tially. The main idea behind this modification is to return the es-timate of the source signal immediately after the new measure-ment vector is provided. In doing so, we can process the blockseamlessly without waiting for the reception of whole block ofobservations. First, instead of Kalman smoother, we employ theKalman filter to conduct the operations from (15) to (19) in a for-ward direction. In order to ensure the real-time processing, weneed to use multiple Kalman filters, where each Kalman filtercorresponds to single iteration of the EM algorithm. For thesake of simplicity, we here consider two Kalman filters as anexample3. In the first Kalman filter, we do not know the signalexistence vector so that we set and run theKalman filter. Once and are obtained, we replace thecomputation of and in (26) and (27) by an auto-regres-sive update rule

(31)

(32)

3This setup corresponds to single EM iteration.

Fig. 3. Real-time implementation of the sKTS algorithm.

where is a forgetting factor controlling the speed of update.Note that by using (31) and (32) instead of (26) and (27), we cancompute the approximation of and on the fly wheneverthe new measurement vector is available. Once and areobtained, we next identify the signal existence indication vectorusing the greedy tree search described in Section II.D. Using

the newly obtained estimate , the second Kalman filter gener-ates . By multiplying this and , we get the final estimateof . Note that it is straightforward to employ more than twoKalman filters to produce better estimate of . To distinguishthis from the original sKTS algorithm, in the sequel, we refer itto as real-time sKTS (RT-sKTS) algorithm. The block diagramof the RT-sKTS algorithm is depicted in Fig. 3.

H. Convergence BehaviorSince the proposed sKTS algorithm is derived based on the

EM algorithm, we can study the convergence of the EM al-gorithm to understand the behavior of the sKTS algorithm. Ingeneral, the convergence of the EM algorithm can be shownin two steps. The first step is to show that the likelihood func-tion is non-decreasing function of the iteration.Using the standard analysis of the EM algorithm, one can showthat the sKTS algorithm satisfies this property. The second stepis to show that converges to a stationary point of the like-lihood function . Since we do not use the contin-uous model for (the parameter vector is binary), unfor-tunately, it is not easy to prove this convergence property. Nev-ertheless, we show from numerical experiments that the sKTSalgorithm finds an accurate estimate of in a small number ofiterations.

III. APPLICATION TO WIRELESS CHANNEL ESTIMATION

In this section, we study the application of the proposedscheme to the training-based channel estimation problem inwireless communication systems. In many communicationsystems, estimation of channels is done before the symbol de-tection since the channel estimate is required for the detectionof the transmitted symbols. Also, to perform reliable precodingand user scheduling in the transmitter, accurate estimate ofthe channel vector should be fed back from the receiver to thetransmitter. Since the wireless channels whose delay spread islarger than the number of significant paths are well modeledas a sparse signal vector in a discretized delay domain, the CStechniques have been used in the sparse channel estimationproblem [26]–[28]. While existing approaches perform the


sparse channel estimation using only a single observationvector or multiple observation vectors under the assumptionthat the CIR vector is invariant in the block, the proposedsKTS algorithm exploits the simultaneously sparse structure oftime-domain CIR, which matches well with physical character-istics of multi-path fading channels. In this section, we describethe application of the proposed method to the channel estima-tion problem in the orthogonal frequency division multiplexing(OFDM) and the single carrier wireless systems.

A. OFDM SystemsWe first consider the channel estimation problem of the

OFDM systems. In our simulations, we focus on the scenariowhere the number of the pilot symbols transmitted per OFDMsymbol is much smaller than the length of the CIR, therebyforming underdetermined systems in the estimation of the CIR.Note that this scenario will be prevalent when a large number oftransmit antennas are deployed (e.g., in large-scale multi-inputmulti-output systems) since the required number of the pilotsignals is proportional to the number of the transmit antennas.Since too much pilot overhead will eat out the resources andeventually limit the throughput of the systems, it is desirable toestimate the channel with small number of resources. However,when the number of the pilot signals is small, conventionalchannel estimators do not perform well due to the lack ofobservations. Whereas, by exploiting the sparsity structure ofthe CIR vector, the sKTS algorithm overcomes the shortageof pilot signals. In the proposed scheme, as shown in Fig. 4,we randomly allocate the pilot signals in time and frequencyaxis to make the better conditioned system matrix . As aresult, while the support of is invariant for several OFDMsymbols, the composite system matrix is varying per symbol.Let , and be the total number of the subcarriers, the

number of the pilot subcarriers, and the length of the time-do-main CIR, respectively. Further, let be the number of pilotsubcarriers per OFDM symbol. Then the relationship betweenthe pilot signal vector and the observed signal vector

of the OFDM system is expressed as

(33)(34)

where is the vector representing frequency-domain channelresponse. Using the DFT matrix whose entryis given by , the frequency-domain channel response isexpressed in terms of the time-domain CIR as

(35)

where is the vector representing the time-domainCIR and is the matrix that selects the rows ofdepending on the location of pilot subcarriers, i.e.,

... (36)

Recall that are the pilot subcarrier indicesat the th OFDM symbol and is the matrix choosingthe first columns of , i.e.,

(37)

Fig. 4. Pilot allocation in (a) conventional systems (comb-type assignment)and (b) proposed sKTS algorithm.

Using (34) and (35), we obtain the observation model for whichthe proposed sKTS scheme can be readily applied

(38)

where .

B. Single Carrier SystemsIn the single carrier (SC) transmission system, the known

training symbols are sent from the transmitter to the receiverbefore the transmission of the data symbols. Suppose that thelength of the training symbols being transmitted is , then thereceived signal at time is expressed as

(39)

where is the th tap of the CIR at time and is the thtraining symbol. Using a vector notation, we have

(40)


TABLE IIISIMPLIFIED ALGORITHM FOR CHANNEL ESTIMATION IN SC SYSTEMS

where and .To estimate the channel vector , recursive least square (RLS)or Kalman channel estimators have been popularly used [29],[30]. When the training period is small, the system becomesill-posed and thus the estimation quality of these channel esti-mation algorithms would be severely degraded.Whereas, by ex-ploiting the simultaneously sparse structure of and treating

as a system matrix, the sKTS algorithm generates the reli-able estimate of . Note that since the system matrix is a rowvector in (40), each step of the algorithm can be simplified(see Table III).

IV. SIMULATIONS AND DISCUSSION

In this section, we study the performance of the proposedsKTS algorithm. We first conduct simulations with the syn-thetic data, and then we test the performance of sKTS algorithmin wireless channel estimation and MRI image reconstructionproblem. In our simulations, we compare the performance ofthe following algorithms:• Proposed sKTS algorithm: we set the tree search parameter

to 5. Only two iterations with andare performed.

• Conventional Kalman smoother: standard Kalmansmoother [25] is used.

• Oracle-based Kalman smoother: Kalman smoothing is per-formed under the perfect knowledge on the support of theCIR. This algorithm provides the best achievable perfor-mance bound of the proposed sKTS algorithm.

• OMP algorithm [28]: greedy projection is used to estimatethe signal support.

• KSBL algorithm [15]: the SBL algorithm [13] is extendedto sequential signal estimation based on the autoregressivemodel.

• RW1L-DF algorithm [17]: the reweighted optimizationis modified to perform sequential dynamic filtering.

As a metric to measure the estimation performance, we use thenormalized mean square error (MSE) defined as

(41)

A. Experiments With Synthetic Data

1) Simulation Setup: We first evaluate the performance ofthe sKTS algorithm using the synthetic data generated from thesignal model in (14). Here we assume that and are givenby and . The location of nonzero el-ements of is fixed over the interval of and changesrandomly for different intervals. Note that slow variation of sup-port is not considered in the simulations. The signal dimension

Fig. 5. The MSE performance as a function of SNR with (a) and (b).

is set to 200 and the sparsity order is set to 15. Entriesof the matrix are generated from i.i.d. Gaussian distribution

.2) Simulation Results: Fig. 5 presents the MSE performance

of signal recovery algorithms as a function of the SNR. InFig. 5(a) and (b), the auto-regressive parameter is set to 0.8and 0, respectively. Note that when , each element ofis temporally uncorrelated. For both scenarios, the dimension

of the measurement vector is set to 40. As shown in thefigure, the proposed sKTS algorithm outperforms competingalgorithms and also performs close to the Oracle-based Kalmansmoother for the whole SNR regime under test. Since the di-mension of the measurement vector is much smaller than thedimension of the source signal , it is no wonderthat the conventional Kalman smoother does not performwell. Since the OMP algorithm uses each measurement vectorindependently, its performance is also not appealing.Next, we investigate the MSE performance when the system

matrix does not change over the processing block. Since


Fig. 6. The MSE performance when is fixed.

Fig. 7. The MSE performance of the sKTS and KSBL as a function of thenumber of iterations.

the system matrix is fixed, if is ill-conditioned, recovery al-gorithm suffers severe performance loss. In order to alleviatethis phenomenon, we increase the dimension of the measure-ment vector to 60 and also use the iteration control by setting

, and (that is, we performthree iterations with different sparsity parameters). Fig. 6 showsthe MSE performance as a function of SNR.While the sKTS al-gorithm suffers considerable performance loss when is set to45, it performs well when becomes 65.We next take look at the convergence behavior of the sKTS

algorithm and the KSBL algorithm. In this test, we set andto 45 and 0.8, respectively. As shown in Fig. 7, the sKTS al-

gorithm performs close to the Oracle-based Kalman smootherafter two iterations while the KSBL algorithm requires five iter-ations until the performance converges. Due to this reason, eventhough the computational complexity of the greedy tree searchis a bit higher than the complexity of M-step in the original EMalgorithm, overall complexity of the approaches are more or lesssimilar. In fact, our numerical experiments demonstrate that it

TABLE IVPARAMETERS OF THE OFDM SYSTEMS

Fig. 8. The performance of the recovery algorithms for the -sparse channelmodel: (a) MSE and (b) BER performance.

takes 15.69 seconds for the sKTS algorithm to finish recoveryprocess while the time required for the KSBL is 14.97 seconds.

B. Experiments in Application to Channel EstimationIn this subsection, we study the performance of the proposed

scheme in application to the channel estimation of an OFDMsystem.


Fig. 9. The BER performance as a function of SNR for the four distinctive scenarios of (a) EVA channel with and , (b) EVA channel withand , (c) EPA channel with and , and (d) EPA channel with and .

1) Simulation Setup: The specific parameters for the OFDMsystem are summarized in the Table IV. In generating pilotsymbols, we use the quadrature phase shift keying (QPSK)pseudo-random sequence. The pilot signals are transmittedfor every three OFDM symbols. For the OFDM symbol con-taining pilots, we assign pilot symbols. As described in theprevious section, the location of pilot subcarriers is randomlychosen for each OFDM symbol. The remaining OFDM re-sources are filled with data symbols. The binary informationbits are encoded using the half rate convolutional code withgeneration polynomials (171, 133) and the coded bits aremodulated to QPSK symbols. Each code block contains 23 360coded bits. Considering the maximum channel delay spreadspecified in the Table IV, we set the dimension of to200. In generating the complex Rayleigh fading frequency-se-lective channels, we use Jake’s model [31], where temporalcorrelation of the CIR taps for given Doppler frequency(Hz) is expressed as , where is the intervalbetween consecutive pilot symbols in time ( ms)and is the zero-th orderBessel function. For convenience, we use Doppler rate defined

as Doppler frequency normalized by pilot transmission rate,i.e., . We use two types of channel models: 1) the exact-sparse channel model where the location of the nonzero

taps is randomly chosen for every block of OFDM symbolsand 2) the practical channel models specified by 3GPP LongTerm Evolution (LTE) standard [33]. Note that the channeltaps in the standard LTE channel model are only approxi-mately sparse. In order to determine the parameters of theGauss-Markov process and for a given , we minimizethe approximation error between the Gauss-Markov processand the Jake’s model as suggested in [32]. Using the CIRestimates obtained by the sparse signal recovery algorithms,the transmitted symbols are detected by the MMSE equalizerin frequency domain. Then, the channel decoder is used todecode the information bits. As a final metric to evaluate theperformance of the recovery algorithms, we use bit error rate(BER) at the output of the channel decoder.2) Simulation Results: We test the performance of the

channel estimators when the exact -sparse channels are used.The sparsity order for these channels is set to 8 and thedimension of the measurement vector is set to 32. Note that


Fig. 10. The MSE performance of the RT-sKTS algorithm.

when , the pilot resources occupy 3.12% of the overallOFDM resources. We assume that the sparsity structure remainsunchanged over the block of pilot containing OFDMsymbols. We set the Doppler rate to 0.05. In Fig. 8(a) and(b), we plot the MSE and BER performance of the recoveryalgorithms as a function of SNR. From the figure, we clearlyobserve that the sKTS algorithm performs best among allalgorithms under test and also performs close to that of theOracle-based Kalman smoother.We next investigate the performance of the proposed sKTS

algorithm when the practical LTE channel models are used. Inthis test, we observe the behavior of the algorithms for fourdistinctive scenarios: a) EVA channel with and

, b) EVA channel with and , c) EPAchannel with and , and d) EPA channelwith . We set and forEVA and EPA channel models since the EVA channel exhibitslonger delay spread. In Fig. 9, we observe that the sKTS algo-rithm maintains the performance gain over the competing algo-rithms for wide range of Doppler rates. Note that when com-pared to the results of the exact -sparse channel model, wesee that the performance gap between the sKTS and KSBL is abit reduced.Next, we compare the performance of the RT-sKTS described

in Section II.G with the original sKTS algorithm. In this simula-tions, we set and . For the RT-sKTS algorithm,we set . In order to test the performance in a harsh con-dition, we arbitrarily change the delay structure of the CIR forevery 30 observation vectors. To ensure the convergence of theonline update strategy in (31) and (32), we use the first 10 ob-servation vectors for warming up purpose and then use the restfor measuring the MSE performance. Note that in practice, suchwarming up period might not be necessary since the support ofchannel vector would not be changed abruptly in many real ap-plications. In Fig. 10, we see that the RT-sKTS algorithm per-forms close to the original sKTS algorithm in low andmid rangeSNR regime. In the high SNR regime, however, the RT-sKTSalgorithm suffers slight performance loss due to the approxima-tion step of and . Nevertheless, as shown in Figs. 10 and

Fig. 11. A sequence of cardiac MRI images.

Fig. 12. The MSE performance of the dynamic MRI image as a function ofimage sequence index.

8(a), the RT-sKTS algorithm maintains the performance gainover the conventional channel estimators.

C. Experiments in Dynamic MRI ApplicationIn this subsection, we investigate the performance of the

sKTS algorithms in the reconstruction of the dynamic MRIimages. In our test, we use a sequence of 32 32 dimensionalcardiac images shown in Fig. 124We generate themeasurementsby performing two dimensional discrete wavelet transform(DWT) with a 2-level Daubechies-4 wavelet, applying twodimensional DFT matrix and taking the randomly chosenfrequency-domain image samples. We set

. After adding the Gaussian noise to the image, werecover the original image using the recovery algorithms. Wecould empirically observe that the location of nonzero coeffi-cients in wavelet image is slowly changing (i.e., support changeoccurs for only a few places), which matches well with oursimultaneous sparse signal model. In order to capture the mostof signal energy, we set for all images5. In Fig. 12, weplot the MSE of the recovery algorithms for image sequence.The sKTS algorithm outperforms the basis pursuit denoising(BPDN) [1] and RW1L-DF [17] and also performs close to the

4These images are decimated from the original 128 128 images [34]. Theraw image data is available online (http://www.ece.iastate.edu/~namrata/soft-ware.html).

5Actually, is set to the number of coefficient containing 99.9% of the signalenergy.


Oracle-based Kalman smoother. Note that we could not includemodified CS scheme in [34] in our numerical experiments sincelarge number of measurement samples is required for the firstimage.

V. CONCLUSION

In this paper, we studied the problem to estimate thetime-varying sparse signals when the sequence of the correlatedobservation vectors are available. In many signal processingand wireless communication applications, the support of sparsesignals changes slowly in time and thus can be well modeledas simultaneously sparse signal. We proposed a new sparsesignal recovery algorithm for this scenario, referred to assparse Kalman tree search (sKTS), that identifies the supportof the sparse signal using multiple measurement vectors. Theproposed sKTS scheme performs the Kalman smoothing toextract the a posteriori statistics of the source signals and thegreedy tree search to identify the support of the signal. From thecase study of sparse channel estimation problem in orthogonalfrequency division multiplexing (OFDM) and image recon-struction in dynamic MRI, we demonstrated that the proposedsKTS algorithm is effective in recovering the dynamic sparsesignal vectors.

APPENDIX ADERIVATION OF (13)

From (7) and (12), we get

(42)

(43)

(44)(45)

where and are the terms independent of . Using theproperty of the trace, i.e., ,we have

(46)

APPENDIX BDERIVATION OF (24)

Denoting as the transpose of the th row vector of ,we can express the left-hand term of (24) as

(47)

Since and, we further have

(48)

(49)

and hence we finally have

(50)

ACKNOWLEDGMENT

The authors would like to thank the anonymous reviewersfor their valuable comments and suggestions that improved thequality of the paper.

REFERENCES[1] E. J. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles:

Exact signal reconstruction from highly incomplete frequency infor-mation,” IEEE Trans. Inf. Theory, vol. 52, pp. 489–509, Feb. 2006.

[2] J. A. Tropp and A. C. Gilbert, “Signal recovery from random measure-ments via orthogonal matching pursuit,” IEEE Trans. Inf. Theory, vol.53, pp. 4655–4666, Dec. 2007.

[3] D. Needell and J. A. Tropp, “CoSaMP: Iterative signal recovery fromincomplete and inaccurate samples,” Appl. Comput. Harmon. Anal.,vol. 26, pp. 301–321, 2009.

[4] W. Dai and O. Milenkovic, “Subspace pursuit for compressivesensing signal reconstruction,” IEEE Trans. Inf. Theory, vol. 55, pp.2230–2249, May 2009.

[5] J. Wang, S. Kwon, and B. Shim, “Generalized orthogonal matchingpursuit,” IEEE Trans. Signal Process., vol. 60, pp. 6202–6216, Dec.2012.

[6] S. Kwon, J. Wang, and B. Shim, “Multipath matching pursuit,” IEEETrans. Inf. Theory, vol. 60, pp. 2986–3001, May 2014.


[7] D. P. Wipf and B. D. Rao, “Sparse Bayesian learning for basis selec-tion,” IEEE Trans. Signal Process., vol. 52, pp. 2153–2164, Aug. 2004.

[8] S. Ji, Y. Xue, and L. Carin, “Bayesian compressed sensing,” IEEETrans. Signal Process., vol. 56, pp. 2346–2356, June 2008.

[9] J. A. Tropp, A. C. Gilbert, and M. J. Strauss, “Algorithms for simulta-neous sparse approximation. Part I: Greedy pursuit,” Signal Process.,vol. 86, pp. 572–588, 2006.

[10] S. F. Cotter, B. D. Rao, K. Engan, and K. Kreutz-Delgado, “Sparse so-lutions to linear inverse problems with multiple measurement vectors,”IEEE Trans. Signal Process., vol. 53, pp. 2477–2488, Jul. 2005.

[11] J. A. Tropp, A. C. Gilbert, and M. J. Strauss, “Algorithms for simul-taneous sparse approximation. Part II: Convex relaxation,” SignalProcess., vol. 86, pp. 589–602, 2006.

[12] D. P. Wipf and B. D. Rao, “An empirical Bayesian strategy for solvingthe simultaneous sparse approximation problem,” IEEE Trans. SignalProcess., vol. 55, pp. 3704–3716, Jul. 2007.

[13] Z. Zhang and B. D. Rao, “Sparse signal recovery with temporally cor-related source vectors using sparse Bayesian learning,” IEEE J. Sel.Topics Signal Process., vol. 5, pp. 912–926, Sept. 2011.

[14] Z. Zhang and B. D. Rao, “Sparse signal recovery in the presence ofcorrelated multiple measurement vectors,” in Proc. ICASSP, Dallas,TX, USA, 2010, pp. 3986–3989.

[15] R. Prasad, C. R. Murphy, and B. D. Rao, “Joint approximately sparsechannel estimation and data detection in OFDM systems using sparseBayesian learning,” IEEE Trans. Signal Process., vol. 62, no. 14, pp.3591–3603, Jul. 2014.

[16] A. S. Charles, M. Asif, J. Romberg, and C. Rozwell, “Sparsity penaltiesin dynamical system estimation,” in Proc. Conf. Inf. Sci. (CISS), Mar.2011, pp. 1–6.

[17] A. S. Charles and C. J. Rozell, “Re-weighted dynamic filteringfor time-varying sparse signal estimation,” [Online]. Available:http:crarxiv.org/abs/1208.0325

[18] N. Vaswani, “Kalman filtered compressed sensing,” in Proc. ICIP, SanDiego, CA, USA, 2008, pp. 893–896.

[19] J. Ziniel and P. Schniter, “Dynamic compressive sensing oftime-varying signals via approximate message passing,” IEEE Trans.Signal Process., vol. 61, pp. 5270–5284, Nov. 2013.

[20] N. Kalouptsidis, G. Mileounis, B. Babadi, and V. Tarokh, “Adaptivealgorithms for sparse system identification,” Signal Process., vol. 91,pp. 1910–1919, Aug. 2011.

[21] D. Angelosante, S. Roumeliotis, and G. Giannakis, “Lasso-Kalmansmoother for tracking sparse signals,” in Proc. Asilomar Conf., Nov.2009, pp. 181–185.

[22] J. W. Choi, T. J. Riedl, K. Kim, A. C. Singer, and J. C. Preisig, “Adap-tive linear turbo equalization over doubly selective channels,” IEEE J.Ocean. Eng., vol. 36, pp. 473–489, Oct. 2011.

[23] A. P. Demster, N. M. Laird, and D. B. Rubin, “Maximum likelihoodfrom incomplete data via the EM algorithm,” J. R. Statist. Soc. B, vol.39, pp. 1–38, 1977.

[24] J. W. Choi, K. Kim, T. J. Riedl, and A. C. Singer, “Iterative estima-tion of sparse and doubly-selective multi-input multi-output (MIMO)channel,” in Proc. Asilomar Conf., Nov. 2009, pp. 620–624.

[25] T. Kailath, A. H. Sayed, and B. Hassibi, Linear Estimation. Engle-wood Cliffs, NJ, USA: Prentice-Hall, 2000.

[26] W. U. Bajwa, J. Haupt, A. M. Sayeed, and R. Nowak, “Compressedchannel sensing: A new approach to estimating sparse multipath chan-nels,” Proc. IEEE, vol. 98, pp. 1058–1076, Jun. 2010.

[27] C. R. Berger, S. Zhou, J. C. Preisig, and P.Willett, “Sparse channel esti-mation formulticarrier underwater acoustic communication: From sub-space methods to compressed sensing,” IEEE Trans. Signal Process.,vol. 58, pp. 1708–1721, Mar. 2010.

[28] S. F. Cotter and B. D. Rao, “Sparse channel estimation via matchingpursuit with application to equalization,” IEEE Trans. Commun., vol.50, pp. 374–377, Mar. 2002.

[29] T. K. Akino, “Optimum-weighted RLS channel estimation for rapidfading MIMO channels,” IEEE Trans. Wireless Comm., vol. 7, pp.4248–4260, Nov. 2008.

[30] M. B. Loiola, R. R. Lopes, and J. M. T. Romano, “Modified Kalmanfilters for channel estimation in orthogonal space-time coded systems,”IEEE Trans. Signal Process., vol. 60, pp. 533–538, Jan. 2012.

[31] W. C. Jakes, Microwave Mobile Communications. New York, NY,USA: Wiley, 1975.

[32] K. E. Baddour and N. C. Beaulieu, “Autoregresive modeling forfading channel simulation,” IEEE Trans. Wireless Commun., vol. 4,pp. 1650–1662, Jul. 2005.

[33] 3GPP TS 36.101. User Equipment (UE) Radio Transmission and Re-ception, 3rd Generation Partnership Project; Technical SpecificationGroup Radio Access Network (E-UTRA) [Online]. Available: http://www.3gpp.org

[34] N. Vaswani and W. Lu, “Modified-CS: Modifying compressivesensing for problems with partially known support,” IEEE Trans.Signal Process., vol. 58, no. 9, pp. 4595–4607, Sept. 2010.

Jun Won Choi (M’10) received the B.S. and M.S.degrees in electrical and computer engineering,Seoul National University and earned Ph.D. degreein electrical and computer engineering, Universityof Illinois at Urbana-Champaign, respectively. In2010, he joined Qualcomm, San Diego USA andparticipated in wireless communication system/al-gorithm design for commercializing LTE andLTE-A modem chipsets. Since 2013, he has beena faculty member of the Department of ElectricalEngineering, Hanyang University and is leading

signal processing and optimization research group. His research area includeswireless communications, signal processing, optimization, data analytic, andmachine learning.

Byonghyo Shim (SM’09) received the B.S. andM.S.degrees in control and instrumentation engineeringfrom Seoul National University, Korea, in 1995 and1997, respectively. He received the M.S. degree inmathematics and the Ph.D. degree in electrical andcomputer engineering from the University of Illinoisat Urbana-Champaign (UIUC), USA, in 2004 and2005, respectively.From 1997 and 2000, he was with the Depart-

ment of Electronics Engineering, Korean Air ForceAcademy as an Officer (First Lieutenant) and an

Academic Full-time Instructor. From 2005 to 2007, he was with Qualcomm,Inc., San Diego, CA, USA, as a Staff Engineer. From 2007 to 2014, he was withthe School of Information and Communication, Korea University, Seoul, as anAssociate Professor. Since September 2014, he has been with the Departmentof Electrical and Computer Engineering, Seoul National University, where heis presently an Associate Professor. His research interests include wirelesscommunications, statistical signal processing, estimation and detection, com-pressive sensing, and information theory.Dr. Shimwas the recipient of the 2005M. E. VanValkenburg Research Award

from the Electrical and Computer Engineering Department of the Universityof Illinois and 2010 Hadong Young Engineer Award from IEIE. He is cur-rently an Associate Editor of the IEEE WIRELESS COMMUNICATIONS LETTERS,JOURNAL OF COMMUNICATIONS AND NETWORKS, and a Guest Editor of theIEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS (JSAC).

Date post:	02-Nov-2020
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

6136 …islab.snu.ac.kr/upload/statistical_recovery_of_block... · 2018. 8. 13. · 6136...

Documents