Hindawi Publishing CorporationJournal of Computer Networks and CommunicationsVolume 2012, Article ID 601287, 13 pagesdoi:10.1155/2012/601287
Research Article
Block Least Mean Squares Algorithm over Distributed WirelessSensor Network
T. Panigrahi,1 P. M. Pradhan,2 G. Panda,2 and B. Mulgrew3
1 Department of ECE, National Institute of Technology, Rourkela 769008, India2 School of Electrical Sciences, Indian Institute of Technology, Bhubaneswar 713002, India3 Institute for Digital Communication, The University of Edinburgh, Edinburgh EH899AD, UK
Correspondence should be addressed to T. Panigrahi, [email protected]
Received 31 October 2011; Accepted 1 December 2011
In a distributed parameter estimation problem, during each sampling instant, a typical sensor node communicates its estimateeither by the diffusion algorithm or by the incremental algorithm. Both these conventional distributed algorithms involvesignificant communication overheads and, consequently, defeat the basic purpose of wireless sensor networks. In the present paper,we therefore propose two new distributed algorithms, namely, block diffusion least mean square (BDLMS) and block incrementalleast mean square (BILMS) by extending the concept of block adaptive filtering techniques to the distributed adaptation scenario.The performance analysis of the proposed BDLMS and BILMS algorithms has been carried out and found to have similarperformances to those offered by conventional diffusion LMS and incremental LMS algorithms, respectively. The convergenceanalyses of the proposed algorithms obtained from the simulation study are also found to be in agreement with the theoreticalanalysis. The remarkable and interesting aspect of the proposed block-based algorithms is that their communication overheads pernode and latencies are less than those of the conventional algorithms by a factor as high as the block size used in the algorithms.
1. Introduction
A wireless sensor network (WSN) consists of a groupof sensors nodes which perform distributed sensing bycoordinating themselves through wireless links. Since thenodes operate in a WSN function with limited battery power,it is important to design the networks with a minimum ofcommunication among the nodes to estimate the requiredparameter vector [1, 2]. In the literature, a number ofresearch papers have appeared which address the energyissues of sensor networks. According to the energy estimationscheme based on the 4th power loss model with Rayleighfading [3], the transmission of 1 kb of data over a distanceof 100 m, operating at 1 GHz using BPSK modulationwith 10−6 bit-error rate, requires 3 J of energy. The sameenergy can be used for executing 300 M instructions in a100 MIPS/watt general purpose processor. Therefore, it is ofgreat importance to minimize the communication amongnodes by maximizing local estimation in each sensor node.
Each node in a WSN collects noisy observations relatedto certain desired parameters. In the centralized solution,
every node in the network transmits its data to a centralfusion center (FC) for processing. This approach has thedisadvantage of being nonrobust to the failure of the FC andalso needs a powerful central processor. Again the problemwith centralized processing is the lack of scalability and therequirement for a large communication resource [1]. If theintended application and the sensor architecture allow morelocal processing, then it would be more energy efficientcompared to communication extensive centralized process-ing. Alternatively, each node in the network can functionas an individual adaptive filter to estimate the parameterfrom the local observations and by cooperating with theneighbors. So there is a need to search for new distributedadaptive algorithms to reduce communication overhead forlow-power consumption and low-latency systems for real-time operation.
The performance of distributed algorithms depends onthe mode of cooperation among the nodes, for example,incremental [4, 5], diffusion [6], probabilistic diffusion [7],and diffusion with adaptive combiner [8]. To improve therobustness against the spatial variation of signal-to-noise
2 Journal of Computer Networks and Communications
ratio (SNR) over the network, recently an efficient adaptivecombination strategy has been proposed [8]. Also a fullydistributed and adaptive implementation to make individualdecisions by each node in the network is dealt with in [9].
Since in block filtering technique [10], the filter coeffi-cients are adjusted once for each new block of data in contrastto once for each new input sample in the least mean square(LMS) algorithm, the block adaptive filter permits fasterimplementation while maintaining equivalent performanceas that of widely used LMS adaptive filter. Therefore, theblock LMS algorithms could be used at each node in orderto reduce the amount of communications.
With this in mind, we present a block formulation ofthe existing cooperative algorithm [4, 11] based on the dis-tributed protocols. Distinctively, in this paper, the adaptivemechanism is proposed in which the nodes of the sameneighborhood communicate with each other after processinga block of data, instead of communicating the estimatesto the neighbors after every sample of input data. As aresult, the average bandwidth for communication amongthe neighboring nodes decreases by a factor equal to theblock size of the algorithm. In real-time scenarios, thenodes in the sensor network follow a particular protocolfor communication [12–14], where the communication timeis much more than the processing time. The proposedblock distributed algorithm provides an excellent balancebetween the message transmission delay and processingdelay, by increasing the interval between two messagesand by increasing the computational load on each nodein the interval between two successive transmissions. Themain motivation here is to propose communication-efficientblock distributed LMS algorithms (both incremental anddiffusion type). We analyze the performance of the proposedalgorithms and compare them with existing distributed LMSalgorithms.
The reminder of the paper is organized as follows. InSection 2, we present the BDLMS algorithm and its networkglobal model. The performance analysis of BDLMS and itslearning characteristics obtained from a simulation studyare presented in Section 3. Performance analysis of theBILMS and its simulation results are presented in Section 4.The performance of the proposed algorithms in terms ofcommunication cost and latency is compared with theconventional distributed adaptive algorithms in Section 5.Finally, Section 6 discusses the conclusions of the paper.
2. Block Adaptive Distributed Solution
Consider a sensor network with N number of sensornodes randomly distributed over the region of interest. Thetopology of a sensor network is modeled by an undirectedgraph. Let G be an undirected graph defined by a set of nodesV and a set of edges E . Nodes i and j are called neighborsif the are connected by an edgey that is, (i, j) ∈ E . We alsoconsidered a loop which consists of a set of nodes i1, i2, . . . , iNsuch that the node ik is ik+1’s neighbor, k = 1, 2, . . . ,N , andi1 is iN ’s neighbor. Every node in the network i ∈ V isassociated with noisy output di to the input data vector ui.We have assumed that the noise is independent of both input
and output data; therefore, the observations are spatiallyand temporally independent. The neighborhood of node iis defined as the set of nodes connected to node i which isdefined as Ni = j | (i, j) ∈ E [15].
Now, the objective is to estimate an M × 1 unknownvector w◦ from the measurements of N nodes. In orderestimate this, every node is modeled as a block adaptivelinear filter where each node updates its weights using theset of errors observed in the estimated output vector, andbroadcasts that to its neighbors. The estimated weight vectorof the kth node at time n is denoted as wk(n). Let uk(n) bethe input data of kth node at time instant n, then the inputvector to the filter at time instant n is
The corresponding desired output of the node for the inputvector uk(n) is modeled as [16, 17]
dk(n) = uTk (n)w◦ + υk(n), (2)
where υk(n) denotes a temporally and spatially uncorrelatedwhite noise with variance σ2
υ,k.The block index j is related to the time index n as
n = jL + i, i = 0, 1, 2, . . . ,L− 1, j = 1, 2, . . ., (3)
where L is the block length. The jth block contains timeindices n = [ jL, jL + 1, . . . , jL + L − 1]. Combining inputvectors of kth node for block j to form a matrix given by
Xjk =
[
uk(
jL)
, uk(
jL + 1)
, . . . , uk(
jL + L− 1)]T , (4)
the corresponding desired response at jth block index of kthnode is represented as
djk =
[
dk(
jL)
,dk(
jL + 1)
, . . . ,dk(
jL + L− 1)]T
. (5)
Let ejk represent the L× 1 error signal vector for jth block of
kth node and is defined as
ejk = dk
(
j)−X
jkw
jk, (6)
where wjk estimated weight vector of the filter when jth block
of the data is input at the kth node and of the order of M×1.The regression input data and corresponding desired
responses are distributed across all the nodes and are rep-resented in two global matrices:
Xjbg = col
{
Xj1, X
j2, . . . , X
jN
}
,
djbg = col
{
dj1, d
j2, . . . , d
jN
}
.(7)
The objective is to estimate the M×1 vector w from the abovequantities, those collected the data across N nodes. By usingthis global data, the block error vector for the whole networkis
ejbg = d
jbg −X
jbgw. (8)
Journal of Computer Networks and Communications 3
Now, the vector w can be estimated by minimizing MSEfunction as
minw
E∥
∥
∥dbg −Xbgw∥
∥
∥
2. (9)
The time index is dropped here for simple mathematicalrepresentation. Since the quantities are collected data acrossthe network in block format; therefore, the block meansquare error (BMSE) is to be minimized. The BMSE is givenby [17, 18]
BMSE = 1L
[
E[
dTbgdbg
]
− E[
dTbgXbg
]
w − wTE[
XTbgdbg
]
−wTE[
XTbgXbg
]
w]
.
(10)
Let the input regression data u be Gaussian and definedby the correlation function r(l) = σ2α|l| in the covariancematrix, where α is the correlation index and σ2 is the varianceof the input regression data, then the relation betweencorrelation and cross-correlation quantities among blockedand unblocked data can be denoted as [10]
RbgX = LR
gU , R
bgdX = LR
gdu, R
bgd = LR
gd, (11)
where RbgX =E[XT
bgXbg], RbgdX=E[XT
bgdbg], and Rbgd =E[dT
bgdbg],which are the autocorrelation and cross-correlation matricesfor global data in blocked form. Similarly, the correlationmatrices for unblocked data are defined as R
gU = E[UT
g Ug],
Rgdu = E[UT
g dg], and Rgd = E[dT
g dg] where the globaldistribution of data across the network is represented as Ug =[u1, u2, . . . , uN ]T and dg = [d1, d2, . . . , dN ]T . These relationsare also valid for node data in individual nodes.
Now, the block mean square error (BMSE) in (10) isreduced to
BMSE = 1L
[
LRgd − LR
gduw − LwTR
gdu − LwTR
guw]
= Rgd − R
gduw − wTR
gdu −wTR
guw = MSE.
(12)
Comparing (12) with the MSE of conventional LMS forglobal data [17, 19], it can be concluded that the MSE in boththe cases is same. Hence, block LMS algorithm has similarproperties as that of the conventional LMS algorithm. Now,(9) for blocked data can be reduced to a form similar to thatof unblocked data as
minw
E∥
∥
∥dg −Ugw∥
∥
∥
2. (13)
The basic difference between blocked and unblocked LMSlies in the estimation of the gradient vector used in theirrespective implementation. The block LMS algorithm usesa more accurately estimated gradient because of the timeaveraging. The accuracy increases with the increase in blocksize. Taking into account the advantages of block LMS overconventional LMS, the distributed block LMS is proposedhere.
2.1. Adaptive Block Distributed Algorithms. In adaptive blockLMS algorithm, each node k in the network receives theestimates from its neighboring nodes after each block ofinput data to adapt the local changes in the environment.Two different types of distributed LMS in WSN have beenreported in literature, namely, incremental and diffusionLMS [6, 19]. These algorithms are based on conventionalLMS for local learning process which in terms needs largecommunication resources. In order to achieve the sameperformance with less communication resource, the blockdistributed LMS is proposed here.
2.1.1. The Block Incremental LMS (BILMS) Algorithm. In anincremental mode of cooperation, information flows in asequential manner from one node to the adjacent one inthe network after processing one sample of data [4]. Thecommunications in the incremental way of cooperation canbe reduced if each node need to communicate only afterprocessing a block of data. For any block of data j, it is
assumed that node k has access to the wjk−1 estimates from
its predecessor node, as defined by the network topologyand constitution. Based on these assumptions, the proposedblock incremental LMS algorithm can be stated by reducingthe conventional incremental LMS algorithm ((16) in [19])to a blocked data form as follows,
wj0 = w j−1,
wjk = w
jk−1 +
μkL
L−1∑
q=0
uk(
jL + q)
×(
dk(
jL + q)− uT
k
(
jL + q)
wjk−1
)
= wjk−1 +
μkL
X j Tk
(
djk −X
jkw
jk−1
)
, for k = 1, 2, . . . N ,
w j = wjN ,
(14)
where μk is the local step size, and L is the block size.
2.1.2. The Block Diffusion LMS (BDLMS) Algorithm. Here,each node k updated its estimate by using a simple localrule based on the average of its own estimates plus theinformation received from its neighbor Nk. In this case, forevery jth block of data at the kth node, the node has accessto a set of estimates from its neighbors Nk. Similar to blockincremental LMS, the proposed block diffusion strategy fora set of local combiners ckl and for local step size μk can bedescribed as a reduced form of conventional diffusion LMS[6, 20] as
θj−1k =
∑
l∈Nk, j−1
cklwj−1k , θk(−1) = 0,
wjk = θ
j−1k +
μkL
L−1∑
q=0
uk(
jL + q)
×(
dk(
jL + q)− uT
k
(
jL + q)
θj−1k
)
.
(15)
4 Journal of Computer Networks and Communications
The weight update equation can be rewritten in morecompact form by using the data in block format given in (4)and (5) as
wjk = θ
j−1k +
μkL
Xjk
T(
djk −X
jkθ
j−1k
)
. (16)
Comparing (15) with (19) in [21], it is concluded that theweight update equation is modified into block format.
3. Performance Analysis of BDLMS Algorithm
The performance of an adaptive filter is evaluated in terms ofits transient and steady-state behaviors, which, respectivelyprovide the information about how fast and how well afilter is capable to learn. Such performance analysis is usuallychallenging in interconnected network because each node kis influenced by local data with local statistics {Rdx,k,RX ,k}, byits neighborhood nodes through local diffusion, and by localnoise with variance σ2
υ,k. In case of block distributed system,the analysis becomes more challenging as it has to handledata in block form. The key performance metrics used inthe analysis are MSD (mean square deviation), EMSE (excessmean square error), and MSE for local and also for globalnetworks and are defined as
ηjk = E
∥
∥
∥wjk−1
∥
∥
∥
2(MSD),
ζjk = E
∥
∥
∥eja,k
∥
∥
∥
2(EMSE),
ξjk = E
∥
∥ek( j)∥
∥2 = ζ
jk + σ2
υ,k (MSE),
(17)
and the local error signals such as weight error vector and apriori error at kth node for jth block are given as
wjk−1 = w◦ − w
jk−1,
eja,k = u
jkw
jk−1.
(18)
The algorithm described in (15) is looking like the intercon-nection of block adaptive filters instead of conventional LMSadaptive algorithm among all the nodes across the network.As shown in (12) that the block LMS algorithm has similarproperties to those of the conventional LMS algorithm, theconvergence analysis of the proposed block diffusion LMSalgorithm can be carried out similar to the diffusion LMSalgorithm described in [18, 21].
The estimated weight vector for jth block across thenetwork is defined as
w j =[
wj1; . . . ; w
jN
]
. (19)
Let C be the N × N metropolis with entries [ckl], then theglobal transaction combiner matrix G is defined as G = C ⊗IM . The diffusion global vector for jth block is defined as
θ j = Gw j . (20)
Now, the input data vector at jth block is defined as
X j = diag{
Xj1, . . . , X
jN
}
. (21)
The desired block responses at each node k are assumedwhich have to obey the traditional data model used inliterature [16–18], that is,
djk = X
jkw◦ + v
jk, (22)
where vjk is the background noise vector of length L. The
noise is assumed to be spatially and temporarily independentwith variance σ2
υ,k. Using blocked desired response for singlenode (17), the global response for kth block can be modeledas
djbg = X
jbgw◦
g + v j , (23)
where w◦g is the optimum global weight vector defined for
every node and is written as w◦g = [w◦; , . . . , ; w◦] and
v j =[
vj1; , . . . , ; v
jN
]
(LN × 1) (24)
is the additive Gaussian noise for jth block index.Using the relations defined above, the block diffusion
strategy in (15) can be written in global form as
w j = θ j−1 +1L
SX j(
djbg −X jθ j−1
)
, (25)
where the step sizes for all the nodes are embedded in amatrix S,
S = diag{
μ1IM ,μ2IM , . . . ,μN IM}
(NM ×NM). (26)
Using (20), it can be written as
w j = Gw j−1 +1L
SX j(
djbg −X jGw j−1
)
. (27)
3.1. Mean Transient Analysis. The mean behavior of theproposed BDLMS is similar to diffusion LMS given in [18,21]. The mean error vector signal is given as
E[
w j]
=(
INM − 1L
SRX
)
GE[
w j−1]
, (28)
where RX = diag{RX ,1, RX ,2, . . . , RX ,N} is a block diagonalmatrix and
RX ,k = E[
XkTXk
]
= LE[
UTk Uk
]
= LRU. (29)
Hence, (28) can be written as
E[
w j]
= (INM − SRU)GE[
w j−1]
. (30)
Comparing (30) with that of diffusion LMS ((35) in [21]),we can find that both block diffusion LMS and diffusion LMSyield the same characteristic equation for the convergence ofmean; and it can be concluded that block diffusion protocoldefined in (15) has the same stabilizing effect on the networkas diffusion LMS,
Journal of Computer Networks and Communications 5
3.2. Mean-Square Transient Analysis. The variance estimateis a key performance indicator in mean-square transientanalysis of any adaptive system. The variance relation forblock data is similar to that of conventional diffusion LMS
E∥
∥
∥w j∥
∥
∥
2
Σ= E
∥
∥
∥w j−1∥
∥
∥
2
Σ′+
1L2
E[
v j TX jSΣSX j Tv j]
, (31)
Σ′ = GTΣG− 1LGTΣSE
[
X j TX j]
G
− 1LGTE
[
X j TX j]
SΣG
+1L2
GTE[
X j TX j]
SΣSE[
X j TX j]
G.
(32)
Using E[X jTX j] = LE[U jTU j] from the definition in (32),we obtain
Σ′ = GTΣG−GTΣSE[
U j TU j]
G−GTE[
U(
j)TU j
]
SΣG
+ GTE[
U j TU j]
SΣSE[
U jTU j]
G,
(33)
which is similar to (45) in [21]. Using the properties ofexpectation and trace [18], the second term of (31) is solvedas
1L2
Ev j TX jSΣSX j Tv j = 1L2
E[
tr[
X jSΣSX j T]]
E[
v j Tv j]
= E[
n j TU jSΣSU j Tn j]
,
(34)
where the noise variance vector n j is not in block form, andit is assumed that the noise is stationary Gaussian. Equations(31) and (32) may therefore be written as
E∥
∥
∥w j∥
∥
∥
2
Σ= E
∥
∥
∥w j−1∥
∥
∥
2
Σ′+
1L2
En j TU jSΣSU j Tn(
j)
, (35)
Σ′ = GTΣG−GTΣSE(
U j TU j)
G−GTE(
U j TU j)
SΣG
+ GTE(
U j TU j)
SΣSE(
U jTU j)
G.
(36)
It may be noted that variance estimate (36) for BDLMSalgorithm is exactly the same as that of DLMS [21]. In theblock LMS algorithm, the local step size is chosen to be Ltimes that of the local step size of diffusion LMS in orderto have the same level of performance. As the proposedalgorithm and the diffusion LMS algorithm have similarproperties, the evolution of their variances is also similar.Therefore, the recursion equation of the global variances forBDLMS will be similar to (73) and (74) in [21]. Similarly,the local node performances will be similar to (89) and (91)of [21].
3.3. Learning Behavior of BDLMS Algorithm. The learningbehavior of BDLMS algorithm is examined using simula-tions. The characteristic or variance curves are plotted for
0 20 40 60
0
20
40
60
1
2
34
5
6
7
8
λ/2 unit
λ/2
un
it
−20
−40
−60−20−40−60
Figure 1: Network topology used for block diffusion LMS.
block LMS and are compared with that of DLMS. The rowregressors with shift invariance input [18] are used with eachregressor having data as
In block LMS, the regressors for L = 3 and M = 3 are givenas
Xk(1) =
⎛
⎜
⎜
⎝
uk(1) 0 0
uk(2) uk(1) 0
uk(3) uk(2) uk(1)
⎞
⎟
⎟
⎠
,
Xk(2) =
⎛
⎜
⎜
⎝
uk(4) uk(3) uk(2)
uk(5) uk(4) uk(3)
uk(6) uk(5) uk(4)
⎞
⎟
⎟
⎠
.
(38)
The desired data are generated according to the modelgiven in literature [18]. The unknown vector w◦ is set to[1, 1, . . . , 1]T/
√M.
The input sequence {uk(i)} is assumed to be spatiallycorrelated and is generated as
uk(i) = ak · uk(i− 1) + bk · nk(i), i > −∞. (39)
Here, ak ∈ [0, 1) is the correlation index, and nk(i) isa spatially independent white Gaussian process with unit
variance and bk =√
σ2u,k · (1− a2
k) . The regressors power
profile is given by {σ2u,k} ∈ (0, 1]. The resulting regressors
have Toeplitz covariance with corelation sequence rk(i) =σ2u,k · (ak)|i|, i = 0, 1, 2, . . . ,M − 1.
Figure 1 shows an eight-node network topology usedin the simulation study. The network settings are given inFigures 2(a) and 2(b).
6 Journal of Computer Networks and Communications
1 2 3 4 5 6 7 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1C
orre
lati
on in
dex-σ k
Node k
(a)
1 2 3 4 5 6 7 80
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Pow
er p
rofi
le-σ
2 u,k
Node k
(b)
Figure 2: Network statistics used for the simulation of BDLMS. (a) Network corelation index per node. (b) Regressor power profile.
0 1000 2000 3000 4000 5000
0
Diffusion LMSBDLMS
Stea
dy-s
tate
MSD
(dB
)
−5
−10
−15
−20
−25
−30
−35
−40
−45
−50
Block number i
Figure 3: Global mean-square deviation (MSD) curve for diffusionand block diffusion LMS.
3.4. The Simulation Conditions. The algorithm is valid forany block of length greater than one [10], while L = M isthe most preferable and optimum choice.
The background noise is assumed to be Gaussian whitenoise of variance σ2
υ,k = 10−3, and the data used in the studyis generated using dk(n) = uk(n)w◦ + vk(n). In order to gen-erate the performance curves, 50 independent experimentsare performed and averaged. The results are obtained byaveraging the last 50 samples of the corresponding learningcurves. The global MSD curve is shown in Figure 3. This is
obtained by averaging E‖w j−1k ‖2 across all the nodes over
100 experiments. Similarly, the global EMSE curve obtained
by averaging E‖eja,k‖2, where ej
a,k = xjkw
j−1k , across all the
nodes over 100 experiments is displayed in Figure 4. The
Stea
dy-s
tate
EM
SE (
dB)
0 1000 2000 3000 4000 5000
0
Diffusion LMSBDLMS
−5
−10
−15
−20
−25
−30
−35
−40
−45
−50
Block number i
Figure 4: Global excess mean-square deviation (EMSE) curve fordiffusion and block diffusion LMS.
global MSE is depicted in Figure 5. It shows that in both thecases the MSE is exactly matching.
Since the weights are updated and then communicatedfor local diffusion after every L data samples, the numberof communications between neighbors is reduced by L timescompared to that of the diffusion LMS case where the weightsare updated and communicated after each sample of data.
The global performances are the contributions of allindividual nodes, and it is obtained by taking the meanperformance of all the nodes. The simulation results areprovided to compare with that obtained by diffusion LMS forindividual node. The local MSD evolution at node 1 is givenin Figure 6(a) and at node 5 is given in Figure 6(b). Similarly,the local EMSE evolution at nodes 1 and 7 is depicted in
Journal of Computer Networks and Communications 7St
eady
-sta
te M
SE (
dB)
0 1000 2000 3000 4000 5000
0
Diffusion LMSBDLMS
−5
−10
−15
−20
−25
−30
−35
Block number i
Figure 5: Global mean-square deviation (MSE) curve for diffusionand block diffusion LMS.
Figure 7. The convergence speed is nearly the same in bothMSD and EMSE evolution, but the performance is slightlydegraded in case of BDLMS. The loss of performance incase of BDLMS could be traded for the huge reduction inof communication bandwidth.
4. Performance Analysis of BILMS Algorithm
To show that the BILMS algorithm has guaranteed conver-gence, we may follow the steady-state performance analysisof the algorithm using the same data model as the one whichis commonly used in the conventional sequential adaptivealgorithms [5, 22, 23].
The weight-energy relation is derived by using thedefinition of weighted a priori and a posteriori error [18]
∥
∥
∥wjk
∥
∥
∥
2
Σ+
∣
∣
∣e jΣa,k
∣
∣
∣
2
∥
∥
∥Xjk
∥
∥
∥
2
Σ
=∥
∥
∥wjk−1
∥
∥
∥
2
Σ+
∣
∣
∣e jΣp,k
∣
∣
∣
2
∥
∥
∥Xjk
∥
∥
∥
2
Σ
. (40)
Since (40) is similar to that of (35) in [19]. Thus, theperformance of BILMS is similar to that of ILMS. Thevariance expression is obtained from the energy relation (40)by replacing a posteriori error by its equivalent expression andthen averaging both the sides
E[∥
∥
∥wjk
∥
∥
∥
2
Σ
]
= E[∥
∥
∥wjk−1
∥
∥
∥
2
Σ′
]
+∣
∣
∣
∣
μkL
∣
∣
∣
∣
2
E[
V jTk X
jkΣX j T
k Vjk
]
Σ′ = Σ− μkL
(
ΣX j Tk X
jk + X j T
k XjkΣ)
+∣
∣
∣
∣
μkL
∣
∣
∣
∣
2
X j Tk X
jkΣX j T
k Xjk.
(41)
The variance relation in (41) is similar to the variancerelation of ILMS in [19]. The performance of ILMS is studied
in detail in literature. It is observed that the theoreticalperformance of block incremental LMS and conventionalincremental LMS algorithms are similar because both havethe same variance expressions. Simulation results provide thevalidation of this analysis.
4.1. Simulation Results of BILMS Algorithm. For the simu-lation study of IBLMS, we have used the regressors withshift-invariance as with the same desired data used in thecase of BDLMS algorithm. The time-correlated sequences aregenerated at every node according to the network statistics.The same network has been chosen here for simulationstudy as defined for block diffusion network in Section 3.3.In incremental way of cooperation, each node receivesinformation from its previous node, updates it by using owndata, and sends the updated estimate to the next node. Thering topology used here is shown in Figure 8. We assumethat the background noise to be temporarily and spatiallyuncorrelated additive white Gaussian noise with variance10−3. The learning curves are obtained by averaging theperformance of 100 independent experiments, generated by5,000 samples in the network. It can be observed from figuresthat the steady-state performances at different nodes of thenetwork achieved by BILMS matche very closely with that ofILMS algorithm. The EMSE plots which are more sensitive tolocal statistics are depicted in Figures 9(a) and 9(b). A goodmatch between BILMS and ILMS is observed from theseplots. In [19], the authors have already proved the theoreticalmatching of steady-state nodal performance with simulationresults. As the MSE roughly reflects the noise power and theplot indicates the good performance of the adaptive network,it may be inferred that the adaptive node performs well in thesteady state.
The global MSD curve shown in Figure 10 is obtained
by averaging E‖ψ( j−1)k ‖2 across all the nodes and over 50
experiments. Similarly, the global EMSE and MSE plotsare displayed in Figures 11 and 12, respectively. These are
obtained by averaging E‖ea,k( j)‖2, where ea,k( j) = xk, j ψ( j−1)k
across all the nodes over 50 experiments.
If the weights are updated after L data points and thencommunicated for local diffusion, the number of communi-cations between neighbors is reduced by L times that of ILMSwhere the weights are updated after processing each sampleof data. Therefore, similar to BDLMS, the communicationoverhead in BILMS also gets reduced by L times that of ILMSalgorithm.
The performance comparison between two proposedalgorithms BDLMS and BILMS for the same network isshown in Figures 13–15. One can observe from Figure 13that the MSE for BILMS algorithm is converging fasterthan BDLMS. Since the same noise model is used for boththe algorithms, therefore after convergence, the steady-stateperformances are the same for both of them. But in case ofMSD and EMSE performances in Figures 14 and 15, littledifference is observed. It is due the different cooperationscheme used for different algorithms. However, the diffusioncooperation scheme is more adaptive to the environmentalchange compared to the incremental cooperation. But
8 Journal of Computer Networks and Communications
Stea
dy-s
tate
MSD
(dB
)
0 1000 2000 3000 4000 5000
0
Diffusion LMSBDLMS
−5
−10
−15
−20
−25
−30
−35
−40
−45
−50
Block number i
(a)
Stea
dy-s
tate
MSD
(dB
)
0 1000 2000 3000 4000 5000
0
Diffusion LMSBDLMS
−5
−10
−15
−20
−25
−30
−35
−40
−45
Block number i
(b)
Figure 6: Local mean-square deviation (MSD) comparison between block diffusion LMS and diffusion LMS. (a) MSD curve at node 1. (b)MSD curve at node 7.
0 1000 2000 3000 4000 5000
0
Diffusion LMSBDLMS
Stea
dy-s
tate
EM
SE (
dB)
−10
−20
−30
−40
−50
10
Block number i
Figure 7: Local EMSE at node 7 for the same network.
a higher number of communication overhead are requiredfor BDLMS than BILMS algorithm.
5. Performance Comparison
In this section, we present an analysis of communicationcost and latency to have a theoretical comparison of theperformances of distributed LMS with block distributedLMS.
1
2
34
5
6
7
8
0 20 40 60
0
20
40
60
λ/2 unit
λ/2
un
it
−20
−40
−60−20−40−60
Figure 8: Network topology.
5.1. Analysis of Communication Cost. Assuming that themessages are of fixed bit width, the communication cost ismodeled as the number of messages transmitted to achievethe steady-state value in the network. Let N be the numberof nodes in the network, and let M be the filter length. Theblock length L is chosen to be the same as the filter length.Let h be the average time required for the transmission of onemessage, that is, for one communication between the nodes[24–26].
Journal of Computer Networks and Communications 9
1 2 3 4 5 6 7 8−30
−29.9
−29.8
−29.6
−29.5
−29.4
−29.3St
eady
-sta
te M
SE (
dB)
ILMS
Node, k
BILMS
−29.7
(a)
1 2 3 4 5 6 7 8
−52
−50
−48
−44
−42
−40
−38
Stea
dy-s
tate
EM
SE (
dB)
ILMS
Node, k
BILMS
−46
−54
(b)
1 2 3 4 5 6 7 8
−34
−32
−30
−26
−24
−22
−20
Stea
dy-s
tate
MSD
(dB
)
ILMS
Node, k
BILMS
−28
−36
−38
−40
(c)
Figure 9: Network nodal performance. (a) Mean-square error (MSE) versus node. (b) excess mean-square error (EMSE) versus node. (c)Mean-square deviation (MSD) versus node.
5.1.1. ILMS and BILMS Algorithms. In the incremental modeof cooperation, every node sends its own estimated weightvector to its adjacent node in a unidirectional cyclic manner.Since at any instant of time, only one node is active/allowedto transmit to only one designated node, the number ofmessages transmitted in one complete cycle is N − 1. LetK be the number of cycles required to attain the steady-state value in the network. Therefore, the total number ofcommunications required to converge the system to steadystate is given by
CILMS = (N − 1)K. (42)
In case of BILMS also, at any instant of time, only one nodein the network is active/allowed to transmit to one designated
follower node, as in the case of ILMS. But, in case of BILMS,each node sends its estimated weight vector to its followernode in the network after an interval of L sample periodsafter processing a block of L data samples. Therefore, thenumber of messages sent by a node in this case is reduced toK/L, and accordingly, the total communication cost is givenby
CBILMS = (N − 1)KL
. (43)
5.1.2. DLMS and BDLMS Algorithms. The diffusion-basedalgorithms are communication intensive. In DLMS mode ofcooperation, in each cycle, each node in the network sendsits estimated information to all its connected nodes in the
10 Journal of Computer Networks and Communications
0 1000 2000 3000 4000 5000
0
ILMSBILMS
Stea
dy-s
tate
MSD
(dB
)
−5
−10
−15
−20
−25
−30
−35
−40
Iteration i
Figure 10: Global mean-square deviation (MSD) curve for incre-mental LMS and block incremental LMS.
0 1000 2000 3000 4000 5000
0
ILMSBILMS
Stea
dy-s
tate
EM
SE (
dB)
−5
−10
−15
−20
−25
−30
−35
−40
Iteration i
−45
Figure 11: Global excess mean-square deviation (EMSE) curve forincremental LMS and block incremental LMS.
network. So the total number of messages transmitted by allthe nodes in a cycle is
c =N∑
i=1
ni, (44)
where ni is the number of nodes connected to the ith node,and the total communication cost to attain convergence isgiven by
CDLMS = cK. (45)
In this proposed block diffusion strategy, the number ofconnected nodes ni and the total size of the messages remain
0 1000 2000 3000 4000 5000
0
ILMSBILMS
Stea
dy-s
tate
MSE
(dB
)
−5
−10
−15
−20
−25
−30
−35
Iteration i
Figure 12: Global mean-square deviation (MSE) curve for incre-mental LMS and block incremental LMS.
0 1000 2000 3000 4000 5000
0
BDLMSBILMS
Stea
dy-s
tate
MSE
(dB
)
−5
−10
−15
−20
−25
−30
−35
Block number i
Figure 13: Global mean-square error curve for BILMS and BDLMS.
the same as that of DLMS. But, in case of BDLMS algorithm,each node distributes the message after L data samples.Therefore the communication is reduced by a factor equalto the block length, and the total communication cost in thiscase is given by
CBDLMS = cK
L. (46)
5.2. Analysis of Duration for Convergence. The time intervalbetween the arrival of input to a node and the time of recep-tion of corresponding updates by the designated node(s)may be assumed to be comprised of two major components.
Journal of Computer Networks and Communications 11
0 1000 2000 3000 4000 5000
0
BDLMSBILMS
Stea
dy-s
tate
MSD
(dB
)
−5
−10
−15
−20
−25
−30
−35
−40
−45
−50
Block number i
Figure 14: Global mean-square deviation curve for BILMS andBDLMS.
0 1000 2000 3000 4000 5000
0
BDLMSBILMS
Stea
dy-s
tate
EM
SE (
dB)
−5
−10
−15
−20
−25
−30
−35
−40
−45
−50
Block number i
Figure 15: Global excess mean-square error curve for BILMS andBDLMS.
Those are processing delay to perform the necessary com-putations in a node to obtain the estimates to be updatedand the communication delay involved in transferring themessage to the receiver node(s). The processing delay willvery much depend on the hardware architecture of the nodesto perform the computation which could be widely varying.But, without losing much of the generality of analysis, we canassume that each node has M parallel multipliers and onefull adder to implement the LMS algorithm. Let TM and TA
be the time required for executing a multiplication and anaddition, respectively. Therefore, the processing delay neededfor single update in LMS is
D = 2TM + (M + 1)TA. (47)
The communication delay is mostly due to the implemen-tation of protocols for transmission and reception, whichremains almost the same for different nodes. The locationof nodes will not have any major contribution to the delayunless the destination node is far apart, and a relay node isrequired to make the message reach the destination. In thisbackdrop, we can assume that the same average delay h isrequired to transfer each message for all receiver-transmitterpairs in the network.
5.2.1. Estimation of Delays for the ILMS and BILMS Algo-rithms. In case of ILMS, the duration of each updating cycleby all the nodes is
ND + (N − 1)h, (48)
and the total duration for convergence of the network is givenas
LILMS = [ND + (N − 1)h]K. (49)
If the same hardware as that of ILMS is used for theimplementation of BILMS, the delay for processing one blockof data is 2MTM + M(M + 1)TA = MD. Then the durationof one cycle of update by the block incremental LMS isN{2MTM + M(M + 1)TA} + (N − 1)h, and the duration ofconvergence of this algorithm is
LBILMS = [NMD + (N − 1)h]KL
. (50)
For L =M, the above expression could be reduced to
LBILMS =[
ND +(N − 1)h
L
]
K. (51)
Comparing (51) with (49), we can find that in BILMS theprocessing delay remains the same as that in ILMS, but thecommunication overhead is reduced by L times.
5.2.2. Estimation of Delays for the DLMS and BDLMS Algo-rithms. Similar to ILMS, it is also assumed here that theupdates of a node reaches all the connected nodes after thesame average delay h. Therefore, the communication delayremains the same as that of ILMS, but in this case, it needsmore processing delay to process the unbiased estimatesreceived from the connected neighboring nodes. The totalcommunication delay in a cycle in this case can be givenby cTA + NTM , where c is the total number of messagestransferred in a cycle given by (44). Now, the total duration ofa cycle in diffusion LMS with the same hardware constraintsis given by
LDLMS = [cTA + NTM + ND + (N − 1)h]K. (52)
12 Journal of Computer Networks and Communications
Table 1: Comparison of performances of distributed algorithms and block distributed algorithms.
Table 2: Numerical comparison of performances of sequential andblock distributed adaptive algorithms.
Parameter ILMS BILMS DLMS BDLMS
Communicationcost
950 95 4500 450
Duration ofconvergence
9.5 s 0.95 s 9.5 s 0.95 s
In case of DBLMS, the total communication delay per cycleis reduced by a factor of L, which can be expressed as
LBDLMS = [cTA + NTM + NMD + (N − 1)h]KL
. (53)
The mathematical expressions of communication cost andlatency for the distributed LMS and the block distributedLMS algorithms are summarized in Table 1. A numericalexample is given in Table 2 to show the advantage ofblock-distributed algorithms over the sequential-distributedalgorithms. The authors have simulated the hardware for8-bit multiplication and addition in TSMC 90 nm. Themultiplication and addition time are found to be TA =10−5 ns,TM = 10−3 ns. We assume the transmission delayh = 10−2 s. Looking at the convergence curves obtained fromthe simulation studies, we can say that the network attainssteady state after 250-input data in DLMS and 50-input datain ILMS case. The filter length M as well as the block size Lare taken to be 10 in the numerical study.
6. Conclusion
We have proposed the block implementation of the dis-tributed LMS algorithms for WSN. The theoretical analysisand the corresponding simulation results demonstrate thatthe performance of the block-distributed LMS algorithmsis similar to that of the sequential-distributed LMS. Theremarkable achievement of the proposed algorithms is thata node requires L (block size) times of less communicationscompared to the conventional sequential-distributed LMSalgorithms. This would be of great advantage in reducingthe communication bandwidth and power consumptioninvolved in the transmission and reception of messagesacross the resource-constrained nodes in a WSN. In thecoming years, with continuing advances in microelectronics,we can accommodate enough computing resources in thenodes to reduce the processing delays in the nodes, butthe communication bandwidth and communication delaycould be the major operational bottlenecks in the WSNs. Theproposed block formulation therefore would have further
advantages over the sequential counterpart in the comingyears.
References
[1] D. Estrin, L. Girod, G. Pottie, and M. Srivastava, “Instrument-ing the world with wireless sensor networks,” in Proceedingsof the IEEE Interntional Conference on Acoustics, Speech, andSignal Processing (ICASP’01), pp. 2033–2036, May 2001.
[2] D. Estin, R. Govindan, J. Heidemann, and S. Kumar, “Nextcentury chanlanges: scalable cordination in sensor network,”in Proceedings of the International Conference on MobileComputing and Networks (MobiCOM ’99), pp. 263–270, 1999.
[3] G. J. Pottie and W. J. Kaiser, “Wireless integrated networksensors,” Communications of the ACM, vol. 43, no. 5, pp. 51–53, 2000.
[4] M. G. Rabbat and R. D. Nowak, “Decentralized sourcelocalization and tracking [wireless sensor networks],” inProceedings of the IEEE International Conference on Acoustics,Speech and Signal Processing, (ICASSP’04), vol. 3, pp. III921–III924, 2004.
[5] A. Nedic and D. P. Bertsekas, “Incremental subgradientmethods for nondifferentiable optimization,” SIAM Journal onOptimization, vol. 12, no. 1, pp. 109–138, 2001.
[6] F. S. Cattivelli and A. H. Sayed, “Diffusion LMS strategies fordistributed estimation,” IEEE Transactions on Signal Process-ing, vol. 58, no. 3, pp. 1035–1048, 2010.
[7] C. G. Lopes and A. H. Sayed, “Diffusion adaptive networkswith changing topologies,” in Proceedings of the IEEE Inter-national Conference on Acoustics, Speech and Signal Processing,(ICASSP’08), pp. 3285–3288, April 2008.
[8] N. Takahashi, I. Yamada, and A. H. Sayed, “Diffusion adap-tive networks with changing topologies,” in Proceedings ofthe International Conference on Acoustics, Speech and SignalProcessing, (ICASSP ’06), pp. 2845–2849, 2009.
[9] F. S. Cattivelli and A. H. Sayed, “Distributed detectionover adaptive networks using diffusion adaptation,” IEEETransactions on Signal Processing, vol. 59, no. 5, pp. 1917–1932,2011.
[10] G. A. Clark, S. K. Mitra, and S. R. Parker, “Block implemen-tation of adaptive digital filters,” IEEE Transactions on SignalProcessing, vol. 29, no. 3, pp. 744–752, 1981.
[11] F. S. Cattivelli and A. H. Sayed, “Analysis of spatial andincremental LMS processing for distributed estimation,” IEEETransactions on Signal Processing, vol. 59, no. 4, pp. 1465–1480,2011.
[12] X. Liu, Q. Huang, and Y. Zhang, “Combs, needles, haystacks:balancing push and pull for discovery in large-scale sensor net-works,” in Proceedings of the Second International Conferenceon Embedded Networked Sensor Systems, (SenSys’04), pp. 122–133, November 2004.
Journal of Computer Networks and Communications 13
[13] S. Madden, M. Franklin, J. Hellerstein, and W. Wong, “TAG:a tiny aggregation service for ad-hoc sensor network,” in Pro-ceedings of the ACM Symposium on Operating System Designand Implementation (OSDI’02), 2002.
[14] J. Ahn and B. Krishnamachari, “Modelling search cost inwireless sensor network,” Tech. Rep. CENG-2007-1, ComputerScience Department, Unversity of southern California, 2007.
[15] A. Jadbabaie, J. Lin, and A. S. Morse, “Coordination of groupsof mobile autonomous agents using nearest neighbor rules,”IEEE Transactions on Automatic Control, vol. 48, no. 6, pp.988–1001, 2003.
[16] B. Widrow and S. D. Strearns, Adaptive Signal Processing,Prentice Hall, Englewood Cliffs, NJ, USA, 1985.
[17] S. Haykin, Adaptive Filter Theory, Prentice Hall, EnglewoodCliffs, NJ, USA, 2001.
[18] A. H. Sayed, Fundamentals of Adaptive Filtering, John Wileyand Sons, 2003.
[19] C. G. Lopes and A. H. Sayed, “Incremental adaptive strategiesover distributed networks,” IEEE Transactions on Signal Pro-cessing, vol. 55, no. 8, pp. 4064–4077, 2007.
[20] D. P. Bertsekas and J. N. Tsitsiklis, “Comments on ”Coordi-nation of groups of mobile autonomous agents using nearestneighbor rules”,” IEEE Transactions on Automatic Control, vol.52, no. 5, pp. 968–969, 2007.
[21] C. G. Lopes and A. H. Sayed, “Diffusion least-mean squaresover adaptive networks: formulation and performance analy-sis,” IEEE Transactions on Signal Processing, vol. 56, no. 7, pp.3122–3136, 2008.
[22] A. Geary and D. P. Bertsekas, “Incremental subgradientmethods for nondifferentiable optimization,” in Proceedings ofthe 38th IEEE Conference on Decision and Control (CDC’99),pp. 907–912, December 1999.
[23] D. P. Bertsekas, “A new class of incremental gradient methodsfor least squares problems,” SIAM Journal on Optimization,vol. 7, no. 4, pp. 913–926, 1997.
[24] R. F. Woods, J. V. McCanny, and J. G. McWhirter, “Frombit level systolic arrays to HDTV processor chips,” Journal ofSignal Processing Systems, vol. 53, no. 1-2, pp. 35–49, 2008.
[25] L.-K. Ting, R. Woods, and C. F. N. Cowan, “Virtex FPGAimplementation of a pipelined adaptive LMS predictor forelectronic support measures receivers,” IEEE Transactions onVery Large Scale Integration (VLSI) Systems, vol. 13, no. 1, pp.86–95, 2005.
[26] P. Fletcher and M. Dean, “Low complexity implementation ofLMS algorithm,” Electronics Letters, vol. 38, no. 15, pp. 836–837, 2002.
