Abstract—This paper studies a novel decomposition technique,suitable for blind separation of linear mixtures of signals com-prising finite-length symbols. The observed symbols are firstmodeled as channel responses in a multiple-input–multiple-output(MIMO) model, while the channel inputs are conceptually con-sidered sparse positive pulse trains carrying the informationabout the symbol arising times. Our decomposition approachcompensates channel responses and aims at reconstructing theinput pulse trains directly. The algorithm is derived first for theoverdetermined noiseless MIMO case. A generalized scheme isthen provided for the underdetermined mixtures in noisy envi-ronments. Although blind, the proposed technique approachesBayesian optimal linear minimum mean square error estimatorand is, hence, significantly noise resistant. The results of simula-tion tests prove it can be applied to considerably underdeterminedconvolutive mixtures and even to the mixtures of moderatelycorrelated input pulse trains, with their cross-correlation up to10% of its maximum possible value.
BLIND source separation (BSS) is becoming an increas-ingly important tool. Over the last decade, it has been
successfully applied to the areas of radar, audio processing,telecommunications, separation of seismic signals, image pro-cessing, and to the analysis of biomedical data [1, pp. 391–448].
Recently, separation of sparse time series has gained a lot ofattention [2], [3]. Assuming the source signals have a sparserepresentation on a given basis, the proposed methods utilizemaximum likelihood (ML) estimators in order to iterativelylearn both the mixing matrix and the source signals out ofthe observed data. They provide reasonably good results, alsofor the underdetermined mixtures. However, by artificiallydividing signals into short blocks and by employing the basisfunctions of popular coding transformations (such as discretecosine transform or wavelet transform) they completely ignorethe time localizations of the underlying signal structures.
Manuscript received May 25, 2006; revised December 15, 2006. This workwas supported in part by the Slovenian Ministry of Higher Education, Scienceand Technology, by the Lagrange Project of CRT Foundation, Italy, and by theMarie Curie Intra-European Fellowship within the 6th European CommunityFramework Programme. The associate editor coordinating the review of thismanuscript and approving it for publication was Prof. Simon J. Godsill.
A. Holobar was with the Faculty of Electrical Engineering and ComputerScience, University of Maribor, 2000 Maribor, Slovenia. He is now with thePolitecnico di Torino, Torino, Italy (e-mail: [email protected]).
D. Zazula is with the Faculty of Electrical Engineering and Computer Sci-ence, University of Maribor, Slovenia (e-mail: [email protected]).
Digital Object Identifier 10.1109/TSP.2007.896108
A more general single-channel generative model, in whichthe signal is described as a linear combination of shift-invariantbasis functions was proposed by Lewicki and Sejnowski [4]and Olshausen [5]. Their methods inherently capture the besttemporal positions of the predefined basis functions and pre-serve the information about the temporal structure of the sig-nals. However, they focus on predefined basis functions only.The idea of shift-invariant basis functions was further extendedby Blumenshath et al. [6], Wersing et al. [7], and Jost et al. [8],who proposed an iterative learning of fundamental signal struc-tures considered as signal-specific basis functions. But they con-strained their search to the set of uncorrelated basis functions.
In this paper, a different approach to sparse identification ofshift-invariant signal components is presented as we address themultichannel linear mixtures of finite-length symbols (corre-lated or not). As explained in the next section, such observa-tions can always be modeled as convolutive mixtures of sparsepulse trains, which carry information about the arising timesof the detected symbols, and the symbols themselves. Insteadof directly estimating the symbols, i.e., convolution kernels, werather focus on the properties of triggering sparse trains. Moreprecisely, we combine their spatial and temporal statistics withthe information about their overlapping probability in order toblindly reconstruct their pulse sequences. The shapes of the ob-served symbols are lost during the decomposition, but can al-ways be recovered by a phase-locked averaging of observations[9]. Throughout this manuscript, we do not assume any priorprobability density function of the modeled pulse trains. We do,however, assume these pulse trains (at most) weakly correlatedand sufficiently sparse, so that the subsequent repetitions of thesame symbol are unlikely to overlap.
The assumed decomposition background is not completelynew. The derived estimator shares almost identical functionalform with the computationally attractive linear minimum meansquare error (LMMSE) estimator, which is Bayesian optimal forlinear mixing systems [10, pp. 379–418]. However, the LMMSEestimator supposes the first two moments of the source signals,i.e., their mean and their cross-correlation with the observa-tions, are known in advance. A supervised way to overcomethis problem was already proposed for interference suppres-sion in the direct sequence (DS) code division multiple access(CDMA) receivers [11]. It starts by a transmission of predefinedsequences of test signals that comprise known source symbols.These test signals are then used on the receiver side to determinethe requested source statistics. Also more sophisticated estima-tion techniques were developed, such as the ML estimator de-scribed in [12, pp. 233–257]. But the ML approach suffers fromhigh computational load and requires at least partial knowledge
4488 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 9, SEPTEMBER 2007
of the mixing matrix [11]. Our approach upgrades the afore-mentioned decomposition techniques by iteratively improvingthe unknown source filter.
The paper is organized in six sections. In Section II, the as-sumed convolutive data model is derived in the form required byour decomposition approach, which is revealed in Sections IIIand IV. Section V presents numerical results obtained by thedecomposition of synthetic mixtures, while a preliminary ap-plication of this method to the real surface electromyogramsand electrocardiograms have been described in [13] and [14],respectively. Section VI concludes the paper.
Throughout this paper, boldface uppercase letters denote ma-trices, boldface lowercase letters denote vectors, while italicsdenote scalars. Discrete time series are denoted by the sub-scripted boldface lowercase letters, e.g.,
, where denotes a single (the th) sample. Thevector of samples taken from time series at the th time in-stant is denoted by . The super-script stands for transpose, while and denote the ma-trix inverse and Moore-Penrose pseudoinverse, respectively.
II. DATA MODEL AND DECOMPOSITION BACKGROUND
Assume different discrete-time observations; given, each comprising
mixtures of up to different samples long symbols
(1)
where stands for the thsymbol, as appearing in the th observation,
with is the pulse trainwhose pulse at time indicates the th repetition of the thsymbol, and stands for the Dirac impulse. We additionallyassume, the subsequent repetitions of the symbol are at least fewsamples apart, while , where
is mathematical expectation. When noisy observations areconsidered, (1) extends to
(2)
where the additive noise is com-monly modeled as a stationary, temporally and spatially whitezero-mean Gaussian random process.
The aforementioned multiple-input–multiple-output (MIMO)data model corresponds to many real world situations and canbe applied whenever the observations can be individually in-terpreted as linear mixtures of separate signal components. Alarge area of application, which has been under intense investi-gation, is most certainly the field of biomedical signals. In the
case of electromyograms, for instance, each symbol corre-sponds to an action potential of the th motor unit as detected bythe th uptake electrode [15]. Similar interpretation can be foundin the case of electrocardiogram, electroneurogram, and even inthe case of electroencephalogram [16]. Another very popularfield of application are digital communication systems. For in-stance, when employing DS-CDMA coding technique, the sym-bols correspond to transmitted information bits modified bythe users’ spreading codes and by the transfer channel responses[12, pp. 849–861]. Further applications include speech recogni-tion, audio separation, stereo image processing, etc.
A. Extension to the Matrix Form
The convolutive relationship described in (1) can always beexpressed in the matrix form [17]. First, the vector of sam-ples related to the th time instant in observations
is extended by delayed repetitionsof each observation to comprise blocks of samples for eachobservation
(3)
Extending the noise vector in thesame way, (1) evolves to
(4)
where the vector
(5)
is an extended form of vector ,which contains a block of consecutive samplesof each , and stands for the mixingmatrix
.... . .
... (6)
which contains the detected symbols enclosed inconvolution kernels, shown in (7) at the bottom of the
page. In the sequel, the th element of vector will be de-noted by , while the sequencewill be referred to as the th extended pulse train.
Assuming the detected symbols are generated in random timeinstants, the pulse trains can be modeled as uncorrelated randompulse sequences. Moreover, when sampling frequency is high
.... . .
. . .. . .
. . .(7)
HOLOBAR AND ZAZULA: MULTICHANNEL BSS USING CONVOLUTION KERNEL COMPENSATION 4489
enough (with respect to the symbol rate), the pulse trains be-come highly sparse. The decomposition approach described inthis paper builds on these assumptions. In particular, the infor-mation contained in the mixing matrix is ignored as we tryto compensate the convolution kernels and focus strictly onthe properties of the pulse trains .
To make it more comprehensible, the description of theproposed convolution kernel compensation (CKC) approach isdivided into two sections. Section III reveals the decompositionunder ideal circumstances, i.e., when the assumed MIMOsystem is overdetermined and noise-free. The underdeterminedcase and the influence of noise are then discussed in Section IV.
III. CONVOLUTION KERNEL COMPENSATION IN A NOISE-FREE
OVERDETERMINED CASE
Suppose the number of symbols is smaller than the numberof observations , and that the extended pulse trains areweakly correlated, i.e., they have small, but significant numberof overlapping pulses. In addition, assume the observationsare ergodic and denote by the correlationmatrix of extended observations. Finally, suppose the extensionfactor is large enough to guarantee ,while thematrix isof full columnrank.Then,bycalculating thesquare of Mahalanobis distance for vector , the convolutionkernels are compensated yielding a so-called activity index
(8)
where stands for correlation matrix of . The activityindex can be thought of as an in-dicator of global pulse train activity. Being always positive, itdiffers from zero only at those time instants where at leastone extended pulse train is active, i.e., ;
.Suppose there are trains active in the observed
time instant and denote them by the set of indices, i.e., . By using a
premultiplying vector instead of in (8), we obtainthe following linear combination of extended pulse trains:
(9)
where stands for the th element of matrix .Now, assume the correlation matrix is diagonally dom-
inant. As proven in Appendix A, its inverse has a superiordiagonal, while all the off-diagonal elements are much smallerthan the diagonal ones. This implies the pulse sequence
has strong contributions only fromthe pulse trains that are contained in , while the contributionsfrom all other trains are much smaller in amplitude. As a result
(10)
According to (10), the entire train can be reconstructed,providing we have found a time instant with a contributionfrom that train only. However, finding such a time instant isnot a trivial task. Moreover, the probability of finding nonover-lapped pulses decreases with the number of extended pulsetrains . Hence, more formal procedure forseparation of superimposed trains (10) is needed.
A. Separation of Superimposed Pulse Trains
Suppose all possible overlaps of pulses from the th andthe th extended pulse train are independent, equally prob-able random events and denote their probability by
. Define andnote that is, at least in the case of weakly correlated pulsetrains, relatively close to zero . Use (10) to recon-struct the pulse sequence ,randomly select a pulse in it and denote by the time ofits occurrence. Now, compute and denote by
the set of indices of all pulse trains withpulses at , i.e., . Because theinstant was chosen according to , there is at least onetrain active in both instants and , i.e., ;
. Generate a sequence of element-wise products, randomly select
pulses in it and denote by ; theirarising times. For each ; there are twocomplementary explanations.
Assumption 1: At least in one train from , a pulseappears at .
Assumption 2: There is no train in with a pulse at, but at least two different trains have pulses at . The first is
contained in set difference , the second in .For each instant ; generate a new se-
quences , denote by the set ofpulse trains active in , and observe the number of pulses inelement-wise product
(11)
The probability that trains are active in all selected timeinstants can be estimated as
(12)
where denotes the cardinal number of . For thesake of simplicity, let and assume that the time instantfulfils Assumption 1, while fulfils Assumption 2. Then the in-tersection containsat leastonepulse train,whileAssumption 2 guarantees that is an empty set.Usinginequality(12),wequicklyrealize theprobabilityofhavingmore than two pulse trains in any intersection ;becomes negligible when . For the same reason, the inter-sections and are,with highprobability, empty sets. As a result, the total number of pulsesin the product (11) can be estimated as (Appendix B, Case 1)
(13)
4490 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 9, SEPTEMBER 2007
where, for the sake of simplicity, we assumed the total numberof pulses for all pulse trains equals .
Now, assume that both selected time instants and fulfilAssumption 1, i.e., the intersectioncomprises at least one pulse train, while (12) guarantees theprobability of having more than one train in this intersectionis negligible. Following the assumptions from the case above,the number of pulses in the product (11) can be estimated as(Appendix B, Case 2)
(14)
According to (13) and (14), the superimposed pulse trains canbe separated by observing the number of pulses in the element-wise product (11), providing that . Whenthis is not the case, the number of components in (11) can beincreased. Generally speaking, the higher the probability , thehigher the optimal value for . However, by increasing the valueof the time complexity of the CKC method also increases. Inall our experiments, proved to be a good compromise.The exact procedure for separation of superimposed trains isdescribed in Section IV.
IV. CONVOLUTION KERNEL COMPENSATION IN
UNDERDETERMINED CASE
Now, assume the extended pulse trains are uncorrelated andequals the identity (note that this assumption is only made
to simplify the theoretical derivations in this section). If thenumber of symbols is greater than the number of observa-tions , the mixing matrix be-comes rectangular and has more columns than rows. As shownin Appendix C, its influence cannot be completely compensatedby (9). What remains is an orthogonal projector :
(15)
where denotes the th element of matrix . We foundexperimentally that when is close to a square matrix, isclose to the identity matrix. By increasing the ratio
, gradually loses its diagonal form and the impact ofthe mixing matrix in (15) increases. Nevertheless, keeping theratio below 2, at least a part of the diagonalin remains dominant (Appendix C).
The pulse trains corresponding to the dominant diagonal el-ements of the matrix still comply with the theory from theoverdetermined case, and the procedure derived in Section IIIcan readily be applied to the underdetermined MIMO systems.This is further confirmed by comparing the LMMSE estimatorto our approach. Namely, employing probabilistic separation ofsuperimposed pulse trains (Section III), the triggering times ofa single, say the th, pulse train can be grouped together into acommon subset of time instants . As-suming the mixing process is ergodic, the observation vectors
can then be averaged over all time instants from the set. The obtained mean vector yields the cross-correlation be-
tween the th pulse train and all the observations
(16)
By inserting (16) into (9), we obtain the LMMSE estimator ofthe th pulse train [10, p. 382]
(17)
A. Noise Reduction
With respect to (16), the influence of noise in (9) can beexpressed as:
(18)
While the first two right-hand side factors in (18) converge tozero when the vector is averaged over a large enough set
, this averaging hardly changes the rightmost factor. Its im-pact can be reduced by truncating the eigenvalues of matrix
. Namely, the influence of noise projected to the space ofinput pulse trains can be estimated as [18, pp. 411–417]
(19)
where stands for the condition number of matrix , andis the noise projection to the space of
extended pulse trains . By setting the smallest singular valuesof to zero, we improve its condition number and increasethe robustness of our decomposition [18, p. 418].
Now, assume the matrix equals the identity matrix. Thenthe conditional number of can be controlled by truncating theeigenvalues of , i.e., by setting the smallesteigenvalues to zero
(20)
where stands for the diagonal matrix with eigenvalues of, sorted in descending order, is a matrix of corresponding
eigenvectors, and denotes the matrix with allelements equal to zero. Afterwards, a new correlation matrix
is constructed and used in (17) in place of. The optimal degree of eigenvalue truncation depends on
the number of symbols, , and the signal-to-noise ratio (SNR),and will be further clarified in Section V.
The final CKC decomposition procedure is described inFig. 1. The noise variance in step 1 can be estimated byobserving the smallest eigenvalues of the matrix [1, p.129], whereas the threshold in step 5 can be computed as aproduct of the observed signal length and the lowest expectedsymbol rate.
V. SIMULATION RESULTS
The proposed decomposition algorithm was tested on threedifferent sets of synthetic signals. The first experiment evalu-ated the influence of number of observed symbols, the secondexperiment studied the influence of pulse overlapping prob-ability , while in the third experiment the decomposition of
HOLOBAR AND ZAZULA: MULTICHANNEL BSS USING CONVOLUTION KERNEL COMPENSATION 4491
Fig. 1. Pseudocode of the proposed CKC decomposition approach.
ill-conditioned linear mixtures was investigated. In all threeexperiments, three different performance measures were ob-served: the number of reconstructed pulse trains, sensitivityof decomposition algorithm, i.e., the percentage of accuratelyidentified pulses per pulse train, and false alarms, i.e., thepercentage of false pulses per reconstructed pulse train. Theproposed decomposition approach was additionally comparedto LMMSE estimator. Recall that the LMMSE estimatorsupposes the cross-correlation vector in (17) known inadvanced. This means the comparison of the decompositionresults obtained by LMMSE and our CKC is feasible just insimulated cases, while in real situations only the separation byCKC approach is implementable.
A. Experiment 1: The Influence of Number of Symbols
The first experiment evaluated the CKC performance in de-pendence of the number of symbols . Fifteen simulation runswere performed, with set equal to 10, 20, and 30 (five runsper each ). In each run, random input pulse trains,
, were generated with the meaninterpulse interval (IPI) set equal to 50 samples and the values
; , uniformly distributed over the interval. In this way, the probability of overlapped pulses, , was
estimated according to . Parameter was fixed to 20samples, yielding pulse overlapping probability of .The length of simulated pulse trains was set equal to 10 000samples. Random zero-mean symbols of length ofsamples were generated and convolved with the simulated pulse
trains to produce the observed signals (1). Note that the repeti-tions of the same symbol in each observation did not interfere.The number of observations was set equal to 25. Finally, fiverealizations of noise per each simulation run and each SNR weresimulated, resulting in 375 different mixtures. These mixtureswere then extended by the empirically selected factorfor and for and . As a re-sult, the number of extended pulse trains increased to 190 for
, 580 for , and 870 for , while thenumber of extended observations was fixed at 250 for ,and at 500 for and , respectively.
Each mixture was decomposed three times, with the degreeof eigenvalue truncation set equal to 0%, 20%, and 40%,respectively. Here means no truncation at all, while
indicates that, according to (20), 40% of the smallesteigenvalues of were set equal to zero. The results, averagedover all simulation runs, are reported in Figs. 2–4, respectively.
In the case of , almost all pulse trains were identi-fied by both estimation techniques (CKC and LMMSE) downto SNR of 0 dB. For and SNR down to 5 dB, the av-erage percentage of pulse trains identified by the CKC methoddropped to 65%, and further decreased to approx. 55% whendealing with 30 symbols . This agrees perfectly withthe theoretical expectations presented in Appendix C. Namely,both cases correspond to the underdetermined convolutive mix-tures, with the ratio set at 1.17 and 1.74, re-spectively. This is also reflected in the performance of LMMSEestimator, which reconstructed only approx. 5% of input trains
4492 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 9, SEPTEMBER 2007
Fig. 2. Average number of recognised pulse trains depending on the number ofsymbols, N , the level of eigenvalue truncation (T ) and SNR. The results arenormalized with respect to the number of symbols N . Note the different scaleon the ordinate.
Fig. 3. The average number of accurately identified pulses per reconstructedpulse train (sensitivity) depending on the number of symbols, N , the level ofeigenvalue truncation (T ) and SNR.
Fig. 4. The average percentage of incorrectly identified pulses per recon-structed pulse train (false alarms) depending on the number of symbols N , thelevel of eigenvalue truncation (T ), and SNR. Note the different scale on theordinate.
more than the CKC method. However, even with a highly under-determined system, the eigenvalue truncation proved to be ben-eficial, increasing the number of reconstructed trains at
by more than 5%, on average. The optimal degree of trun-cation depends strongly on the level of noise. For example, for
the optimal value of yielded 0% atand increased to 20% at .
Fig. 5. The average number of reconstructed pulse trains for different valuesof overlapping probability p versus SNR. The number of symbols was set equalto N = 10.
The CKC method proved to be highly robust. On average,more than 97% of reconstructed pulses were accurately recog-nized down to the SNR of 5 dB, while there was hardly anymisplaced pulse (Fig. 4). This is consistent with the theoreticalderivation in (16), where it was shown that the accuracy of theCKC method matches the accuracy of the LMMSE estimator.In the case of and , however, a slight de-crease in CKC performance was noticed at SNR of 0 dB. Theresults in Fig. 3 also demonstrate a significant positive correla-tion between the sensitivity of CKC method and the degree ofeigenvalue truncation. Separating the mixtures withsymbols at SNR of 0 dB using the 0% of eigenvalue truncation,for example, the sensitivity of CKC method dropped to 97%.Increasing the degree of truncation to 20%, the sensitivity indexincreased back to 98%. At the same time, there was a slight in-crease of false alarms, indicating a possible negative correlationbetween the specificity of the CKC algorithm and the eigenvaluetruncation.
B. Experiment 2: The Influence of Level of Correlation AmongPulse Trains
The second experiment studied the influence of pulse over-lapping probability . Both the number of symbols, , and theextension factor, , were fixed at 10. The number of observa-tions was set equal to 25. Following the simulation protocolfrom the first experiment, 10 000 samples long random pulsetrains were generated. The mean IPI was set at 50 samples, whilethe IPI variability was uniformly distributed over the interval
. In this experiment, parameter was set at 10, 6.6, and 5,yielding pulse overlapping probability of ,and , respectively. Finally, random zero-mean symbols
of length samples were convolved with the simu-lated pulse trains to produce the observed signals (1). The ob-servations were additionally corrupted by additive zero-meanGaussian noise (five realisations of noise per each SNR). Theresults, averaged over five simulation runs per each , are de-picted in Figs. 5–7.
As expected, the performance of the CKC method drops sig-nificantly with the overlapping probability . Each time is in-creased by 0.025, the percentage of reconstructed pulse trainsdrops by approx. 8%. At , only 75% of all the pulse
HOLOBAR AND ZAZULA: MULTICHANNEL BSS USING CONVOLUTION KERNEL COMPENSATION 4493
Fig. 6. The average number of accurately identified pulses per reconstructedpulse train (sensitivity) for different values of pulse overlapping probability pversus SNR. The number of symbols N was set equal to 10.
Fig. 7. False alarm index for different values of pulse overlapping probability,p, versus SNR. The number of symbols, N , was set equal to 10.
trains were reconstructed (with SNR of 10 dB and ).Also the sensitivity decreased, while the false alarms increasedsignificantly (from 0.004% at to 0.009% atfor SNR of 10 dB). There are several possible explanations ofthis phenomenon. The first, and most probable one, relies onthe probabilistic separation of the superimposed pulse trains(Section III). By increasing the probability the beneficial av-eraging effect in (16) decreases. As a result, the impact of bothnoise and superimposed pulses is increased. This also agreeswith the observed increase of the CKC sensitivity to noise. Inaddition, high overlapping probability causes the correlationmatrix (and, hence, its inverse) to become significantly non-diagonal, which additionally increases the devastating impact ofthe superimposed pulses in the reconstructed pulse trains. At thesame time, no significant correlation between the probabilityand the LMMSE estimation was noticed (except maybe in falsealarm index). This was expected because the LMMSE estimatorutilizes prior knowledge on the correlation between the obser-vations and input pulse trains.
C. Experiment 3: Decomposition of Ill-Conditioned Mixtures
The final experiment was conducted on a simulated surfaceelectromyograms (SEMG), based on a planar volume conductormodel [19]. The thickness of the skin, fat and muscle layer wasset equal to 1, 3, and 10 mm, respectively. Imitating the anatomyof skeletal muscles, synchronously active muscle fibers werefirst grouped into so called motor units (MUs) [20]. MUs wererandomly scattered over the detection volume with their size
Fig. 8. Average number of recognised MUs depending on the number of simu-lated MUs, N , the level of eigenvalue truncation, T , and SNR. The results arenormalized with respect to the number of simulated MUs,N . Note the differentscale on the ordinate.
varying from 25 to 2500 fibers. Average semifiber length was setequal to 70 mm. The mean muscle fiber conduction velocity was
. When activated, each MU transmitted detectableelectric potentials. These potentials were additionally low-passfiltered to simulate the effect of biological tissues that separatethe MU from the pick-up electrodes. Biopotentials, which cor-respond to the symbols in our data model, were detectedat the surface of the skin. Because of different MUs depths inthe muscle tissue, the biopotentials differed significantly in am-plitude, yielding the power ratio of up to 10 dB. The averagelength of all detected biopotentials was 12 ms. The numberof pick-up electrodes, , was fixed at 60 (a 2-D electrode gridof 13 5 electrodes with interelectrode distance of 5 mm wassimulated, while the signals were detected in longitudinal singledifferential configuration). Based on a MU recruitment model[21], three different muscle contraction levels were simulatedcorresponding to , , and simultane-ously active MUs. For each MU, a random sequence of innerva-tion pulses was generated. The average motor unit discharge ratewas set equal to pulses per second, while the averageIPI variability equalled 20% of the mean IPI. Finally, SEMGobservations were sampled at 1024 Hz. Before the decomposi-tion took place, these observations were additionally corruptedby additive zero-mean Gaussian noise (five realizations of noiseper each SNR) and extended by the factor forand for and . This resulted in a con-dition number of about for the mixing matrix . The de-composition results, averaged over 15 simulation runs (5 runsper each ), are depicted in Figs. 8–10.
Both the CKC and LMMSE estimator exhibit a significantdrop in the number of reconstructed pulse trains (when com-pared to the results of the first experiment). At and
only 70% of simulated MUs were identifiedby the LMMSE estimator. Under the same conditions the CKCmethod reconstructed only a half of simulated MUs (comparethis to almost 100% reconstruction in Experiment 1). This is notunexpected, as the decomposition of surface electromyogramsis well known to be strongly ill-conditioned. In addition, due tolarge differences among the powers of different MUs, smallerMUs are often missed and considered a background noise. Inspite of these facts, the reconstruction of pulse trains still proved
4494 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 9, SEPTEMBER 2007
Fig. 9. The average number of accurately identified pulses per reconstructedMU (sensitivity) depending on the number of simulated MUs, N , the level ofeigenvalue truncation T , and SNR.
Fig. 10. The average percentage of incorrectly identified firings per recon-structed MU (false alarms) depending on the number of simulated MUs, N ,the level of eigenvalue truncation T , and SNR.
to be highly accurate. Slight decrease of the CKC sensitivity wasnoticed (compared to the results of Experiment 1). At the sametime, a slight increase of false alarms was observed (for both theCKC and LMMSE estimator), but this had no significant influ-ence on decomposition performance.
VI. CONCLUSION
The proposed CKC decomposition method can be applied toa variety of linear mixtures whenever the observations can beinterpreted as compound signals comprising finite-length sym-bols. By compensating the shapes of the detected symbols, it ef-ficiently combines all the assumed statistical properties of theirarising times and operates directly in the space of sparse pulsetrains. The proposed approach is significantly noise resistant,while successfully resolving also the underdetermined convolu-tive mixtures with a relatively large number of input trains.
In this paper, three different experimental settings weretested, evaluating the performance of CKC in the case of under-determined and ill-conditioned mixtures (Experiments 1 and 3),and in the case of weakly correlated pulse trains (Experiment2). Simulation results proved the accuracy of the CKC perfectlymatches the accuracy of the LMMSE estimator. But, contrary tothe LMMSE estimator, the CKC method exhibits a significantnegative correlation between the number of reconstructed pulsetrains and the pulse overlap probability . There are several
possible explanations of this phenomenon, the most probableone including the induced non-diagonality of the correlationmatrix (see the discussion of Experiment 2). Invariance ofthe LMMSE estimator to the pulse overlapping originates fromthe necessity of a prior knowledge of the trains’ cross-corre-lation with the observations. The drop of performance in thecase of correlated pulse trains is the price we have to pay forreconstructing the pulse trains blindly.
Both the CKC and LMMSE estimator exhibit a significant per-formance drop when decomposing ill-conditioned and underde-termined mixtures. Typically, only the strongest observed sym-bols (in the sense of signal energy) are reconstructed, while all theothers are treated as a background noise. In the case of biomed-ical signals, this is not a serious problem as the number of recon-structed input trains is already limited by the detection volumeof the pick-up electrodes [13]. In the case of communication sig-nals, on the other hand, the ratio between the number of observa-tions and the number of users can be improved by increasing thesampling frequency. This also decreases the overlapping prob-ability and, hence, improves the performance of the proposedapproach. The number of reconstructed pulse trains could be in-creased also by subtracting the identified symbol observationsfrom the observed mixtures [22]. However, in the case of noise,this idea proves difficult to implement as it is very hard to ob-tain a perfect cancellation of the identified symbols.
Finally, the proposed decomposition approach reconstructsonly the arising times and shapes of the detected symbols. Inmany cases (e.g., when decomposing biomedical signals), thisends the decomposition, as we are only interested in the detectedform of symbols. When processing the communication signals,however, an additional single-input multiple-output (SIMO) de-composition step is needed in order to compensate the effect ofthe transfer channels and estimate the original source symbolsout of their observations.
APPENDIX AINVERSE OF DIAGONALLY DOMINANT MATRIX
Suppose the pulse train correlation matrix is diagonallydominant, i.e., [18, p. 184]. If the pulsetrains are weakly correlated we also have
(A.1)
The matrix can be written as a sum of its diagonal andnon-diagonal part , whereand . Then,using the Neumann series, the inverse of can be expressedas [18, pp. 126]:
(A.2)
providing the absolute value of each eigenvalue of the ma-trix is smaller than 1. According to Gerschgorin’s the-
HOLOBAR AND ZAZULA: MULTICHANNEL BSS USING CONVOLUTION KERNEL COMPENSATION 4495
orem, the eigenvalues of are contained in a union ofthe circles defined by [18, p. 498]
(A.3)
Therefore, the first-order approximation of yields
, which, whenand (A.1) are taken into account, proves the inverse ofhas a superior diagonal. According to extensive numericalsimulations, similar conclusions also apply to the matriceswhich are not strictly diagonally dominant but still fulfill thecondition in (A.1).
APPENDIX BAVERAGE NUMBER OF PULSES IN THE ELEMENT-WISE
PRODUCT OF PULSE TRAINS
Although the number of components in (11) can be con-sidered an arbitrary value, we only focus on two cases (and ). The derivations for other cases follow those pre-sented here and are left to the interested reader. For the sake ofsimplicity, we also assume the number of pulses in each trainequals .
First, observe the pulses in the element-wise product, . This
product will certainly contain the pulses of all the trains from. Their average number of pulses can be estimated
as where , whilewith ; denotes the correction factorintroduced by the fact that the pulses of the trains inmutually overlap
(B.1)
where denotes the number of combinations of ele-
ments, taken elements at a time. The product will alsocontain all those pulses of any pulse train from the set difference
which randomly overlap with the pulses of any pulsetrain from . Their number can be estimated as
(B.2)
Hence, the average number of pulses in the productyields
(B.3)
We can follow the same route in the case of com-ponents in (11). First, define the following mutually disjunctivesets:
(B.4)
with and , , . Usingdenotation , the average number of pulses in(11) can be estimated as:
(B.5)
where, supposing , all the factors multiplied by to thesecond or higher powers were neglected. According to (B.5),the average number of pulses depends mainly on the number oftrains in the sets , , and and their distributionin the corresponding sets. In the sequel, we are going to studyjust the cases that prove the (13) and (14), respectively.
Case 1: Suppose the set contains a single pulse train,while sets , , and are empty. Fur-ther assume that , , are all equal to or less than 1. Then(B.5) yields
(B.6)
Use andrecall that, on average, the number of pulse trains in the setcan be estimated as , where stands for thenumber of symbols, is extension factor and is the symbollength. Then, (B.6) simplifies to .
Case 2: Suppose the set contains a single pulse train,while sets , , and are empty. Furtherassume that , , are all equal to or less than 1. Then (B.5)simplifies to
(B.7)
APPENDIX CELIMINATION OF UNDERDETERMINED MIXING MATRIX
Define where is an arbitrary mixingmatrix of size , with
. By definition, is orthogonal projector and positivesemidefinite [18, pp. 434]. Denoting by the matrix of the rightsingular vectors of , the th element of matrix can becalculated as
(C.1)
where denotes the th element of the matrix . Consid-ering the orthogonality of matrix and denoting by its thcolumn, three different cases can emerge.
1. The energy of is concentrated in the first ele-ments. This is the ideal case as (C.1) guarantees the diag-
4496 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 55, NO. 9, SEPTEMBER 2007
onal element will be close to 1, while all off-diagonalelements will be close to 0.
2. The energy of is distributed among all elements.This is the most probable case and implies the valueof the diagonal element decreases with the ratio
, while the values of off-diagonalelements simultaneously increase.
3. The majority of ’s energy is concentrated in the lastelements. This is the worst case sce-
nario as (C.1) shows the diagonal element will be closeto zero.
The number of columns being concomitant with case3, is limited. Namely, supposing all the columns of the unitarymatrix correspond to case 3, it would necessarily imply that
is similar to the projection matrix mapping -dimensional space onto -dimensionalsubspace. These arguments further support the following empir-ical observation:
Observation 1: For most , supposing the ratiosmall, the matrix will have at least
a part of its diagonal elements superior to all off-diagonalelements.
The numerical simulations further reveal the dominant diag-onal elements are at least several magnitudes higher than thecorresponding off-diagonal elements, as long as the number ofextended pulse trains, , does not exceed thenumber of extended observations, , by factor 2.
REFERENCES
[1] A. Hyvarinen, J. Karhunen, and E. Oja, Independent Component Anal-ysis. New York: Wiley, 2001.
[2] M. Girolami, “A variational method for learning sparse and overcom-plete representations,” Neural Comput., vol. 13, pp. 2517–2532, 2001.
[3] M. Davies and N. Mitianoudis, “A simple mixture model for sparseovercomplete ICA,” Inst. Elect. Eng. Proc. Vision, Image and SignalProcess., vol. 151, pp. 35–43, 2004.
[4] M. S. Lewicki and T. J. Sejnowski, “Coding time-varying signalsusing sparse, shift invariant representations,” in Advances Neural Inf.Process. Syst., 1999, vol. 11, pp. 730–736.
[5] B. A. Olshausen, R. P. N. Rao, B. A. Olshausen, and M. S. Lewicki,Eds., “Sparse codes and spikes,” in Probabilistic Models of the Brain,2002, pp. 257–272.
[6] T. Blumensath and M. Davies, “Sparse and shift-invariant representa-tions of music,” IEEE Trans. Audio, Speech Language Process., vol.14, pp. 50–57, 2006.
[7] H. Wersing, J. Eggert, and E. Körner, “Sparse coding with invari-ance constraints,” in Proc. ICANN 2003, Istanbul, Turkey, 2003, pp.285–392.
[8] P. Jost, P. Vandergheynst, S. Lesage, and R. Groibonval, “MoTIF: Anefficient algorithm for learning translation invariant dictionaries,” inProc. ICASSP 2006, 2006, vol. 5, pp. V-857–V-860.
[9] C. Disselhorst-Klug, G. Rau, A. Schmeer, and J. Silny, “Non-invasivedetection of the single motor unit action potential by averaging the spa-tial potential distribution triggered on a spatially filtered motor unit ac-tion potential,” J. Electromyogr. Kinesiol., vol. 9, pp. 67–72, 1999.
[10] S. M. Key, Fundamentals of statistical signal processing: Estimationtheory. Englewood Cliffs, NJ: Prentice-Hall Int., 1993.
[11] U. Madhow, “Blind adaptive interference suppression for direct-se-quence CDMA,” Proc. IEEE, vol. 86, pp. 2049–2069, 1998.
[12] J. Proakis, Digital Communications, 3rd ed. New York: McGraw-Hill, 1995.
[13] A. Holobar and D. Zazula, “Correlation-based approach to separationof surface electromyograms at low contraction forces,” Med. Biol. Eng.Comput., vol. 42, pp. 487–496, 2004.
[14] D. Zazula and A. Holobar, “Blind source separation based on a singleobservation,” in Proc. BPC 2006, Setúbal, Portugal, 2006, pp. 76–85.
[15] R. Merletti, “Surface electromyography: Possibilities and limitations,”J. Rehab. Sci., vol. 7, pp. 25–34, 1994.
[16] M. S. Lewicki, “A review of methods for spike sorting: The detec-tion and classification of neural action potentials,” Network: Comput.Neural Syst., vol. 9, pp. 53–78, 1998.
[17] B. Boashash, Time-Frequency Signal Analysis and Processing. En-glewood Cliffs, NJ: Prentice-Hall PTR, 2001.
[18] C. D. Meyer, Matrix Analysis and Applied Linear Algebra. Philadel-phia, PA: SIAM, 2001.
[19] D. Farina and R. Merletti, “A novel approach for precise simulation ofthe EMG signal detected by surface electrodes,” IEEE Trans. Biomed.Eng., vol. 48, pp. 637–646, 2001.
[20] R. Merletti and P. A. Parker, Electromyography: Physiology, Engi-neering and Noninvasive Applications. Hoboken, NJ: Wiley, 2004.
[21] A. J. Fuglevand, D. A. Winter, and A. E. Patla, “Models of recruitmentand rate coding organization in motor unit pools,” J. Neurophysiol., vol.70, pp. 2470–2488, 1993.
[22] T. Ristaniemi and T. Huovinen, “Joint delay tracking and interferencecancellation in DS-CDMA systems using successive ICA for oversat-urated data,” in Proc. ICA2004, Granada, Spain, 2004, pp. 1173–1180.
Ales Holobar (M’98) received the B.S. and D.Sc.degrees in computer science from the University ofMaribor, Slovenia, in 2000 and 2004, respectively.
From 2000 to 2006 he was a researcher at theFaculty of Electrical Engineering and ComputerScience, University of Maribor. He is currently aMarie Curie Fellow at Politecnico di Torino, Torino,Italy. His research interests include virtual reality,conceptual learning, and signal processing, withcurrent activities focused on blind source separationand biomedical signal processing.
Dr. Holobar is a member of ISEK, IAPR, and Slovenian Society of PatternRecognition.
Damjan Zazula (M’87–SM’04) received theDipl.Ing., Master’s, and Doctor of Science degreesin electrical engineering from the University ofLjubljana, Slovenia, in 1974, 1978, and 1990,respectively.
After being involved in industrial R&D for12 years, he joined the Faculty of Electrical En-gineering and Computer Science, University ofMaribor, Slovenia, in 1987. Currently, he holds aFull Professor position in computer science, andfrom 1998 to 2003, he was also appointed an As-
sociate Dean of Research. He has spent several months as visiting professorwith the ETH in Zurich, Switzerland, and Ecole Centrale de Nantes, France.His main research interests are compound signal decomposition, biomedicalimaging, and virtual training tools.
Dr. Zazula is a member of the IEEE Signal Processing Society, EURASIP,IAPR, Slovenian Technical Society, Slovenian Society of Pattern Recognition,and Slovenian Society of Biomedical Engineering.
