Performance Repeatability of Low Power Wireless SensorNetwork Protocols: A Multi Testbed Study

Taewoo KwonComputer Science and

Engineering395 Dreese Hall

The Ohio State UniversityColumbus, OH, 43210, USA

kwonta@cse.ohio-state.edu

Emre ErtinElectrical and Computer

Engineering205 Dreese Labs

The Ohio State UniversityColumbus, OH, 43210, USA

ertin.1@osu.edu

Anish AroraComputer Science and

Engineering395 Dreese Hall

The Ohio State UniversityColumbus, OH, 43210, USA

anish@cse.ohio-state.edu

ABSTRACTPredicting and bounding performance is a fundamental re-quirement in wireless sensor network (WSN) protocol de-velopment. To support a fast development cycle princi-pled methods are required for experimentation with wire-less protocols in testbed environments that can provide re-peatable performance guarantees in the target environment.In this paper we propose a method to achieve performancerepeatability across test and target environments, that re-lies on analytical prediction of expected protocol perfor-mance as a function of RF environment parameters andforwarding protocol. For the validation of the proposedmethod, we present analytical, simulation, and experimentalresults for one-dimensional networks deployed in indoor andoutdoor propagation environments, and for two-dimensionalnetworks on four major indoor WSN testbeds to validatethe performance of the proposed method on achieving re-peatable protocol behavior across diverse set of RF environ-ments.

Categories and Subject DescriptorsC.2.1 [Computer Systems Organization]: Computer-Communications NetworkArchitecture and Design[WirelessCommunication]; C.3 [Computer Systems Organization]:Special-Purpose and Application-Based Systems—Real-timeand Embedded Systems; C.4 [Computer Systems Orga-nization]: Performance of Systems—Modeling Techniques

General TermsAlgorithms, Experimentation, Measurement, Performance

KeywordsWireless Sensor Network,Testbed, Spatial Scaling, ProtocolPerformance Repeatability

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.MSWiM’11, October 31–November 4, 2011, Miami, Florida, USA.Copyright 2011 ACM 978-1-4503-0898-4/11/10 ...$10.00.

1. INTRODUCTIONPredicting and bounding performance is a fundamental re-

quirement in wireless sensor network (WSN) protocol devel-opment. Researchers deploying low-power wireless networkshave observed that the network behavior in the field divergedsubstantially from that seen in testbeds, which made it diffi-cult to provide performance guarantees using testbed experi-ments. To support a fast development cycle principled meth-ods are required for experimentation with wireless protocolsin testbed environments that can provide repeatable perfor-mance in the target environment. In this paper we proposea method to achieve performance repeatability across testand target environments, that relies on analytical modelingof protocol performance as a function of RF environment pa-rameters and adjusting transmit power levels for matchingperformance measured through a user specified metric.

The link quality (e.g. SNR, PRR) is a function of the RFpropagation environment, and makes direct impact on pro-tocol behavior (e.g. link selection), which in turn determinesthe protocol performance. In our previous study [1], we pro-posed a generic feature link usage spectrum for summarizingprotocol behavior. Link usage spectrum is the probabilitydistribution with which the network protocol selects linksof different length from among all the available links in thenetwork at hand. and proposed to adjust power levels formatching the link usage spectrum as close as possible be-tween the two different RF environments. The hypothesis ofthis method is that the link usage spectrum is a gross predic-tor of the performance of (a rich class of) network protocols.With this hypothesis, a network protocol will perform com-parably in two network settings if the respective link usagespectrums of the protocol match closely in these settings.

Contributions of the paper. In this paper, we extendour previous work to provide a generally applicable analyt-ical tool for adjusting transmit power levels for replicatingperformance across the testbed and the deployment envi-ronments by matching any user-specified performance index(e.g. the number of transmissions, packet forwarding dis-tance, delay, goodput, energy efficiency). Broadly speaking,wireless network protocol performance is dependent uponthe quality of the links that are chosen by each node fromits available linkset, in terms of the mean and the varianceof their SNR. Our tool exploits the insight that scalar net-work wide performance metrics (such as expected latency)

393

aggregate link usage probabilities with the conditional linkSNR statistics into a single performance index.Armed with this insight, we derive analytical expressions

for the expected value of a performance metric given theRF environment and the networking protocol. Based on themathematical modeling of link usage spectrum, we deriveclosed form expressions for the moments of the link SNRs,conditioned on the link being chosen by the protocol. Wethen show how the conditional moments of link SNRs andthe link usage spectrum can be used to evaluate the expectedvalue of arbitrary protocol performance metrics.We validate our tool through various simulation and ex-

perimental studies. It is desirable to validate our methodin large scale networks. However it is very difficult to exe-cute such experiments in multiple RF environments. Espe-cially, most of the indoor testbeds can not accommodatelarge topologies. Therefore, we execute and present twocanonical studies: micro and macro study. By combiningthe two, we show the proposed analytical tool will work forgeneral topologies. Micro study uses one-dimensional chaintopology with 20 TelosB nodes. The micro study is designedto adopt more granularity in choice of lengths of links thanmacro study. We executed the macro study in four majorWSN testbeds: KanseiGenie [2], Motelab [3], Tutornet [4],and TWIST [5]. Macro study used two-dimensional gridtopology with 16 TelosB (KanseiGenie) or TmoteSky (Mote-lab, Tutornet, TWIST) nodes.For micro study, we first present simulation studies that

show the validity of the analytical approximations employed.Then, we use outdoor deployment and indoor testbed ex-periments to show that the analytically calculated powerscaling factors result in repeatable average protocol behav-ior and performance across the two environments, despitethe difference in node spacing as well as radio propagationparameters. For macro study, we compare analytically de-rived link usage with the experimentally observed usage infour major WSN testbeds. Then, we compare analyticallypredicted average performance with the experimentally ob-served performance using two metrics (the number of trans-missions and the progress to destination)with data from thethe four testbeds.Road map. In Section 2, we present the background anddiscuss related work. In Section 3, we provide a theoreti-cal study to characterize the wireless protocol performance.In Section 4, we validate the performance of our tool. InSection 4.1, we present micro study and in Section 4.2, weprovide macro study. In Section 5, we summarize our obser-vations and discuss future work.

2. RELATED WORKSpatial scaling is an important problem to achieve the per-

formance repeatability between a testbed and a deployment.Naik et al. studied the spatial scaling problem within thesame RF environment [6]. [6] compared the performancemetrics of two WSN applications in two different scales inthe KanseiGenie testbed [2] by using different transmissionpower. We studied the spatial scaling problem across differ-ent environments [1]. [1] defined the concept of “link usagespectrum”and tried to reproduce the spectrum of the sourcenetwork as closely as possible in the target network. [1] ar-gued that even if the available links to two networks aredifferent but the distribution of the chosen links are compa-rable, the protocol behavior of the two will likely be compa-

rable. This approach is successful in achieving comparablewireless protocol behavior, but does not target a specificperformance metric for predicting and repeating mean per-formance.

Modeling of the expected performance is a general prob-lem for network protocol analysis, and the solution of thisproblem can contribute towards achieving the performancerepeatability. Chiasserini et al. [7] proposed a Markov modelof a sensor network protocol performance whose nodes areduty-cycled. [7] successfully incorporated the interferencemodel of sensor network, but it has a limitation on model-ing realistic data collection protocol, since it assumes staticrouting, whereas in general data collection protocols use dy-namic link selection schemes. Seada et al. and Zuniga etal. [8, 9] proposed PRR× d (the forwarding distance) for ametric for geographic routing and predicted average proto-col performance for optimizing the protocol parameters. [8]and [9] used numerical simulations for estimating averageprotocol performance. [9] quantified the difference betweentheir local optimum metric, PRR×d and the global optimalmetric ETX. They observed that there is only narrow differ-ence between ETX and PRR×d on the energy performance(� r

t), where r is the delivery and t is the total number of

transmissions, for different densities and network sizes.Designing protocols with robust performance across dif-

ferent environments is a related research thrust. For exam-ple, Lin et al. [10] proposed a new link metric called com-petence capturing long term variation of link quality anddesigned a distributed route maintenance framework basedon feedback control systems. [10] tried to achieve the per-formance repeatability by achieving a stable performancethrough choosing more stable links in long term, and equip-ing extra schemes (e.g. Retransmission, Transmission PowerControl) to guarantee certain level of link reliability. In con-trast, our focus is on achieving comparable performance foran arbitrary protocol by tuning the testbed parameters suchas power attenuation and scaling.

3. WIRELESS PROTOCOL PERFORMANCECHARACTERIZATION

In this section we present an analytical tool for deter-mining power scaling to repeat WSN protocol performanceacross testbed and deployment environments. Wireless pro-tocol performance is determined by the subset of links thatare utilized by the wireless protocol and the quality of thesechosen links. The quality of the each link in turn is charac-terized by the RF channel between the terminals (environ-ment model) and the bit-error-performance of their wirelesstransceivers (radio model).

In the following we derive link usage and link qualitystatistics for a given network protocol by combining the RFenvironment and radio receiver models. Then we combinethese statistics to assess the expected performance of thewireless protocol. We consider a wireless network W =({lj}, η, σ) with link set {lj}Mj=1 and the RF environment(η, σ), where η is the path loss exponent (PLE) and σ isthe standard deviation of the received signal strength, em-ploying a network protocol P. We note W is a probabilisticobject, referring to the ensemble of link set realizations. Foreach realization of the wireless networkW, network protocolP chooses a subset of the link set for forwarding of data. Weassume P is a possibly randomized rule for selecting links

394

based on a metric which combines link length c(lj) and linkquality Q(y(lj)) to score the link lj , and the SNR value ofthe link, y(lj). As an example we consider two-dimensionalgrid networks, where the link lengths cj ≡ c(lj) and theprogress to destination dj ≡ d(lj) are constrained to the fi-nite set {(aτ, bτ) : 1 ≤ a, b ≤ N}, where τ is the minimumnode spacing. We index elements of the above finite setfrom 1 to M (=N2). The link usage spectrum L(W,P, i) isthe discrete probability distribution over the length of linksand the progress of links to the destination induced by thenetwork protocol P.

L(W,P, i) = Prob[c(l) = ci, d(l) = di] (1)

, where i is the index of (aτ, bτ), ci =√

(aτ)2 + (bτ)2,

di = ci ∗ cos(|tan−1( ba) − θ|), θ is the angle between source

and destination. The link usage spectrum is a universal sum-mary statistics of the network behavior. The fundamentalimportance of link usage spectrum stems from the fact thatmany network wide metrics can be calculated as the averageover link realizations weighted by the link usage spectrum.The link usage spectrum can be calculated in situ empir-ically as averages over node link usages or can be deriveddirectly from RF environment and physical layer model.

3.1 Calculating the expected performancemetrics

In this subsection, we derive an equation to calculate theexpected performance metric (Result 1). Let f(yi, di) bean arbitrary performance metric over the link quality, SNR(yi), and the progress to destination (di). We assume theperformance of a wireless link is measured by f(yi, di). Thenthe expected performance metric E[f |W,P] is computed asthe average over the linkset li, weighted by the link usagespectrum L(W,P, i) (Result 2):

E[f |W,P] =M∑i=1

E(f(yi, di|li ↑))L(W,P, index(li)) (2)

Conditional moments of the SNR of the chosen links (Re-sult 3) can be used to approximate E(f(yi, di|li ↑)).

Result 1. The expected performance metric, E(f(yi, di|li ↑)), of the link li can be calculated as below:

E(f(yi, di|li ↑)) � f(E(yi|li ↑), di)

+f′′(E(yi|li ↑), di)

2

(E(y2

i |li ↑)− (E(yi|li ↑))2)

, where yi is SNR (dB) values of link li, di is the progressto destination of li

Proof. We review the straightforward argument basedon the Taylor expansion of the function f(y, d) with respectto its first argument.

E[f(yi|li ↑)] = E[f(µyi + (yi − µyi)|li ↑)]Using the Taylor expansion,

f(x) =

n∑k=0

f (n)(a)

n!(x− a)n

, we have

� E[f(µyi |li ↑) + f′(µyi |li ↑)(yi − µyi |li ↑)

+1

2f′′(µyi |li ↑)(yi − µyi |li ↑)2]

� f(E(yi|li ↑)) + f′′(E(yi|li ↑))

2V AR(yi|li ↑)

Finally,

= f(E(yi|li ↑))+

f′′(E(yi|li ↑))

2

(E(y2

i |li ↑)− (E(yi|li ↑))2)

Using Result 1, we can calculate the most common perfor-mance metrics: the expected total number of transmissionsand the expected progress to destination. As an example, bychoosing f(y, d) = 1

PRR(y)×d, we can calculate the expected

number of transmissions per unit distance. By multiplyingthe progress to the destination with the calculated expecteddelay per unit distance, we can calculate the expected end-to-end number of transmissions of the chosen links.

3.2 Calculating the moments of the conditionalSNR

In this subsection, we derive an equation (Result 3) tocalculate the moments of the SNR of a link conditioned onbeing chosen over all other links. We revisit the link usagespectrum derivation (Result 2), which was presented in [1],for further extension in Result 3.

Result 2. For a protocol P which uses PRR × d as themetric for choosing forwarding links, the probability of choos-ing link l of length over all other links is expressed as follows:

L(W,P, index(l))

= P[PRR(y(l))· d(l) = max

i=1...M{PRR(yi)· di}

]

�M−1∑k=0

∫ ak+1

ak

1

σ√2π

[k∏

i=1

(1

2+

1

2erf

(g(βi)− µyi√

2σ

))]

[M∏

i=k+1

(1

2+

1

2erf

(y(l)− µyi√

2σ

))]e−

(y(l)−µy(l))2

2σ2 dy(l),

where βi = di/d(l), ai = PRR−1(min{βi,1βi}), and the links

are enumerated such that a1 ≤ a2 ≤ ... ≤ aM−1, with a0 =−∞, aM =∞ and yi is the SNR experienced by link li, and

g(βi) =

{ai, if βi ≥ 1

∞, if βi < 1

The proof is given in [1].

Result 3. When an application uses PRR×d as the met-ric for choosing forwarding links, the expected value of them-th moment of the SNR of a link l conditioned on the factthat it is chosen by the forwarding metric is given by:

E(y(l)m|l ↑) �

1

w

M−1∑k=0

∫ ak+1

ak

y(l)m

σ√2π

[k∏

i=1

(1

2+

1

2erf

(g(βi)− µyi√

2σ

))]

395

[M∏

i=k+1

(1

2+

1

2erf

(y(l)− µyi√

2σ

))]e−

(y(l)−µy(l))2

2σ2 dy(l)

, where w = L(W,P, index(l)), βi = di/d(l),ai = PRR−1(min{βi,

1βi}), and the links are enumerated

such that a1 ≤ a2 ≤ ... ≤ aM−1, with a0 = −∞, aM = ∞and yi is the SNR experienced by link li, and

g(βi) =

{ai, if βi ≥ 1

∞, if βi < 1

Proof.

E(y(l)m|l ↑) =∫ ∞

−∞y(l)m dy(l)(y(l)|l ↑)dy(l)

From Result 2:dy(l)(y(l)|l ↑)

=1

w

M−1∑k=0

∫ ak+1

ak

1

σ√2π

[k∏

i=1

(1

2+

1

2erf

(g(βi)− µyi√

2σ

))]

[M∏

i=k+1

(1

2+

1

2erf

(y(l)− µyi√

2σ

))]e−

(y(l)−µy(l))2

2σ2 dy(l),

Therefore,E(y(l)m|l ↑)

� 1

w

M−1∑k=0

∫ ak+1

ak

y(l)m

σ√2π

[k∏

i=1

(1

2+

1

2erf

(g(βi)− µyi√

2σ

))]

[M∏

i=k+1

(1

2+

1

2erf

(y(l)− µyi√

2σ

))]e−

(y(l)−µy(l))2

2σ2 dy(l),

3.3 Tuning power for matching performanceIn this subsection, we present a procedure to calculate the

optimal power attenuation to achieve the consistent perfor-mance repetition across two different environments. Equippedwith the analytical expressions for expected network perfor-mance metrics we can define optimal power scaling rule forconsistent performance across testbed and deployment en-vironments. Consider a wireless network W with inter-nodedistances {dj}mj=1 and its scaled version W, with inter-node

distances {dj = αdj}mj=1 in RF environments characterizedby log-normal scale model parameters (n, σ) and (n, σ) re-spectively. We assume that transmit power in the networkW is variable through: Pt = P0+β where β is the power at-tenuation or amplification in the scaled network. Since thescaled vector in general reduces the node distances for con-venient testing, β in general is a negative value indicatingpower attenuation. As a result the scaled network realiza-tions W(β) depends on β1. The optimal power attenuationis defined as:

E[f |W,P] = E[f |W(β),P]1We could also introduce spatial variations in β across thenodes to influence the width of the resulting spectrum, fora better match.

4. VALIDATIONIn this section, we present analytical, simulation, and ex-

perimental studies of repeating protocol performance acrosstestbed and deployed environments. .

Physical & link layerIn our experiments we use 802.15.4 transceivers embeddedin popular sensor network platforms such as TelosB and Mi-caZ. CC2420 radio is compatible with the 2.4GHz 802.15.4standard. 802.15.4 standard wireless physical layer employsblock direct-sequence spread spectrum code with 2MChip/schip rate and250 kbps data rate to achieve processing andcoding gain. The transmitter modulates the carrier usingoffset quadrature phase shift keying (O-QPSK) with half-sine shaping which is equivalent to minimum shift keying(MSK) modulation which has the following Bit Error Rate:

BER = Q(√

2y/PG/CG) =1

2erfc(

√y/PG/CG) (3)

, where y gives the SNR. The processing gain (PG) for802.15.4 is given by 10 log(2/0.25)=9 dB. The coding gain(CG) depends on the increased Hamming distance betweenthe codes and is a function of the SNR itself. For low apacket error rate region coding gain can be approximated as2 dB [11].

Thus, the Packet Reception Rate equation is

PRR =

(1− 1

2erfc(

√x)

)8∗packet size

We note that the bit-error-rate approximation given in Equa-tion 3 assumes coherent demodulation using carrier phase in-formation. Practical transceiver designs use non-zero IF andnoncoherent demodulation. The non-ideal receiver struc-tures can be approximated with SNR reduction or equiva-lently increase in the noise floor (P0) causing only a hori-zontal shift in the PRR curve.We present to studies. The micro study uses a dense set of

nodes in one dimension to provide granularity in the choiceof link lengths. The macro study uses a coarse two dimen-sional topology to provide granularity in the choice of linkdirection towards source.

4.1 Micro study: chain one-dimensionaltopology

4.1.1 Experimental setupWe set up a chain topology with total 20 TelosB sensor

nodes, as done in [1]. Here, we define D2 as the chain topol-ogy with the node separation of 6 ft, and D1 as the sametopology with node separation of 3 ft. Each node is elevatedabout 4 inches from the ground. TelosB mote is equippedwith CC2420 radio and USB serial for communication. Thenode 0 is set to be the source, and the node 19 is set to bethe destination. Every two seconds, the source node pro-duces and sends a packet. For each case, we gathers about1,000 packets generated by the source. We logged all thepaths that each packet has gone through, and only countsthe body parts (i.e. excludes the first and the last hops ofthe paths) to calculate the average link length, because usu-ally the first and the last hops are composed of the shorterlinks than the remainder links of the path. The metric usedfor these experiments are as follows: the average link lengthand the number of transmissions.

396

Analytical prediction of expected performance requiresknowledge of the RF environmental parameters: PLE (η)of indoor and outdoor and standard deviation of RSSI (dB)(σ). We measured the RSSIs of links in a corridor in the sec-ond floor of Dreese Lab and on the top of a parking garagebuilding. For the indoor experiment, we measured RSSIsat 20 different distances (1 ∼ 20 unit, where 1 unit = 3 ft)within the maximum communication range with the highesttransmission power (0 dBm). For the outdoor experiment,10 measurements were taken from the distance of 1 ∼ 10unit distances where 30 ft seems to be the maximum com-munication range with the same transmission power, 0 dBm.Table 1 presents the summary of measured RF propagationenvironment. We also used reported radio sensitivity of -94dBm to adjust for the noise power P0.

Metrics Indoor Outdoor

η 1.7555 2.2776

σ 5.0 dB 4.2 dB

Table 1: Log Normal Model Variables for Indoorand Outdoor RF Environments

4.1.2 Messaging layer and resultsWe test a messaging layer protocol, CTP [12]. CTP is

a tree-based collection protocol. Nodes generate routes toroots using a routing gradient. In micro study, CTP usesETX as the routing metric. ETX implicitly favors long linksover short links because each node selects the path with theminimum number of expected transmissions.Experimental design

TelosB mote uses 2.4 GHz frequency and provides 8 differenttransmission power levels: 31 (0 dBm), 27 (-1 dBm), 23(-3 dBm), 19 (-5 dBm), 15 (-7 dBm), 11 (-10 dBm), 7 (-15 dBm), 3 (-25 dBm) [13]. We also can attach variousattenuators according to attenuation levels (1 dB, 3 dB, andetc.) to modify transmitted and receive power. We used a3 dB attenuator to construct a reasonable communicationenvironment among 20 sensor nodes in a compact indoorarea. We tested 0 dBm for D2, -1 dBm and -6 dBm forD1 at outdoor environment. For scaling node distances byhalf (D2 → D1), while keeping the PLE constant (outdoor)the required transmit power attenuation is give by −10 ×2.2776 × log102 = −6.8514 dB [6]. For indoor tests, weexperimented -18 dBm at 3 ft spacing (D1) to determinethe feasibility of mapping the performance across networkswith different node spacings and PLEs.Results

Figure 1 shows the analytical and Monte-Carlo simulationresults of the link usage spectra for indoor and outdoor en-vironments with specified transmission powers for outdoor(D2 and D1) and indoor (D1) case. We observe outdoor D2at 0 dBm, outdoor D1 at -6 dBm and indoor D1 at -18 dBmproduce similar link usage spectra.Figure 2 shows the analytical average SNR values of each

links chosen over all other links for each test environments.In figure 2, we compare the conditional mean SNR of link oflength d given it was chosen by the forwarding protocol overall other links and the unconditional mean SNR of a linklength d. We observe that E[SNR|df = d] > E[SNR|d].Figure 3 shows the analytical average PRR values of each

links chosen over all other links for each test environments.

0

0.1

0.2

0.3

0.4

0.5

0 5 10 15 20

Usa

ge W

eigh

ts

Link Length

Out:D2 (tx=0dBm)Out:D1 (tx=0dBm)

Out:D1 (tx=-6dBm)In:D1 (tx=-13dBm)In:D1 (tx=-18dBm)

0

0.1

0.2

0.3

0.4

0.5

0 5 10 15 20

Usa

ge W

eigh

ts

Link Length

Out:D2 (tx=0dBm)Out:D1 (tx=0dBm)

Out:D1 (tx=-6dBm)In:D1 (tx=-13dBm)In:D1 (tx=-18dBm)

Figure 1: Link Usage Spectrum. (1) Analytical withResult 2, (2) Monte-Carlo Simulation

-20

-15

-10

-5

0

5

10

15

20

25

0 5 10 15 20

SNR

(dB

)

Link Length

Out:C,D2 (tx=0dB)Out:A,D2 (tx=0dB)Out:C,D1 (tx=0dB)Out:A,D1 (tx=0dB)

Out:C,D1 (tx=-6dB)Out:A,D1 (tx=-6dB)In:C,D1 (tx=-13dB)In:A,D1 (tx=-13dB)In:C,D1 (tx=-18dB)In:A,D1 (tx=-18dB)

Figure 2: SNR of the Chosen/All Links. Analytical(Result 3). C:Chosen, A:All

The conditional mean PRR given the event df = d, is cal-culated by taking the expected conditional SNRs of linksthat are chosen. Again we see the average PRR of the cho-sen links is much higher than the average PRR of all linksin figure 3. (E[PRR|df = d] > E[PRR|d] as discussed inSection 2).

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30 35 40

Pack

et R

ecep

tion

Rat

e

Link Length

Out:C,D2 (tx=0dBm)Out:A,D2 (tx=0dBm)Out:C,D1 (tx=0dBm)Out:A,D1 (tx=0dBm)

Out:C,D1 (tx=-6dBm)Out:A,D1 (tx=-6dBm)In:C,D1 (tx=-13dBm)In:A,D1 (tx=-13dBm)In:C,D1 (tx=-18dBm)In:A,D1 (tx=-18dBm)

Figure 3: PRR of the Chosen/All Links. Analytical(Result 1)

397

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10 12 14 16

Cum

ulat

ive

Usa

ge W

eigh

ts

Distance (3 ft)

I,D1,-18dBmI,D1,-13dBmO,D1,-6dBmO,D1,0dBmO,D2,0dBm

0

0.2

0.4

0.6

0.8

1

0 2 4 6 8 10 12 14 16

Cum

ulat

ive

Usa

ge W

eigh

ts

Distance (3 ft)

I,D1,-18dBmI,D1,-13dBmO,D1,-6dBmO,D1,0dBmO,D2,0dBm

Figure 4: Cumulative Link Usage Spectrum. (1)Analytical (Result 2), (2) Experiment

Figure 4 compares the cumulative link usage spectra an-alytically calculated (1) and experimentally collected (2).Experimental results validate the correctness of the analyt-ical study.

0

5

10

15

20

1 2 3 4 5

Experiments

Num

ber o

f Tra

nsm

issi

ons

O:D20dBm

O:D1−6dBm

O:D10dBm

I:D1−13dBm

I:D1−18dBm

2

4

6

8

10

12

14

1 2 3 4 5

Experiments

Link

Len

gth O:D2

0dBm

O:D10dBm

O:D1−6dBm

I:D1−13dBm

I:D1−18dBm

Figure 5: (1) Average Number of Transmissions(Experimental), (2) Average Link Length (Exper-imental)

Next, we validate analytical predictions in deployment ex-periments using two major performance metrics, link lengthand end-to-end delay. Figure 5(1) shows the average link

length taken from experiments. We can see the strong sim-ilarity between outdoor D2 (0 dBm), D1 (-6 dBm), and in-door D1 (-18 dBm) as predicted with the analytical study(details shown in table 2). Figure 5(2) shows the averageend-to-end delay (the number of transmissions) taken fromexperiments. We also observe the strong similarity betweenthree test cases.

Number of O:D2 O:D1 O:D1 I:D1 I:D1

Transmissions 0dBm 0dBm -6dBm -13dBm -18dBm

Analytical 4.6350 2.4141 4.1452 2.5046 4.3041

Experimental 5.4039 3.2049 5.2481 3.712 5.1616

Table 2: Comparisons of Performance Metrics: End-to-End delay Indoor and Outdoor

Link O:D2 O:D1 O:D1 I:D1 I:D1

Length 0dBm 0dBm -6dBm -13dBm -18dBm

Analytical 4.4085 8.0155 4.9081 7.9520 5.1609

Experimental 4.1735 7.7749 4.1842 6.9329 4.709

Table 3: Comparisons of Performance Metrics: LinkLength Indoor and Outdoor ( Experimental : calcu-lated without the first & last hop)

Table 2 shows the average number of transmissions takenfrom experiments with the specified transmission powers.Table 3 shows the average link lengths taken from experi-ments and compares them with the analytical results. Weassume the topology is infinite to calculate the expected linklength analytically. However, because the topology in exper-iments is limited, the link lengths of marginal hops are usu-ally shorter than those of the intermediate hops. Table 2 and3 show strong similarity in the performance of three match-ing networks (outdoor D2 with 0 dBm, outdoor D1 with -6dBm, and indoor D1 with -18 dBm) in experimental results.This is consistent with the analytically calculated optimalattenuation factors for matching the end-to-end delay per-formance from the outdoor D2 to the outdoor D1 β1 = −6.9dB, from the outdoor D1 to the indoor D1 β2 = −12.3 dBand from the outdoor D2 to the indoor D1 β3 = −19.2 dB.Overall, experimental results show little more transmissionsand little shorter link length than analytically predicted.Because, the last hop to destination should be shorter thanother hops (edge effect), and the first hop usually tends tobe shorter than other hops, we can consider little (about 1)more transmission reasonable.

4.2 Macro study: grid two-dimensionaltopology

4.2.1 Experimental setupWe set up grid topologies in four major WSN testbeds

(KanseiGenie, Motelab, Tutornet, TWIST) with total 16TelosB or TmoteSky sensor nodes. TelosB and TmoteSkymotes are equipped with CC2420 radio and USB serial forcommunication. The node (1,1) is set to be the source, andthe node (4,4) is set to be the destination. Every two sec-onds, the source node produces and sends a packet. For eachexperiment with a specific transmission power, we gathersabout 1,000 packets generated by the source. We logged thepath that each packet has gone through. The metric used for

398

these experiments are as follows: the progress to destinationand the number of transmission.To perform analytical study to be compared with the ex-

perimental results, we measure PLE (η) and standard de-viation of RSSI (σ) in four testbeds and present in table 4.

Metrics KanseiGenie Motelab Tutornet TWIST

η 1.809 3.995 4.800 2.697

σ 5.0 dB 8.5 dB 8 dB 9 dB

Table 4: Log Normal Model Variables for RF Envi-ronments of Testbeds

4.2.2 Messaging layer and resultsWe test a messaging layer protocol, CTP. We customized

CTP, which uses ETX as the default routing metric, to usePRR× d as the routing metric.Experimental design

TelosB and TmoteSky motes use 2.4 GHz frequency andprovide 8 different transmission power levels. We carefullychoose 4×4 (16) nodes, which form grid, in four testbeds.KanseiGenie has natural grid topology and can have themaximum 16×16 grid topology. Motelab and Tutornet donot have natural grid topology, therefore we try to choose16 nodes to form grid-like topology. TWIST has naturalgrid shape topolgoy, but walls divide nodes and inter-nodedistance are not the same for row and column. We testmultiple transmission powers (0 dBm, -5 dBm, -7 dBm, -10 dBm, -15 dBm) to find sets of transmission powers whichmake CTP perform comparably with given topologies in fourtestbeds.Results

To validate Result 2 (calculates the link usage spectrum) inthe two-dimensional topology, we present cumulative usageweights of links with the progress to destination (how muchprogress can be achieved with links). Figure 6(1) shows theanalytical results and figure 6(2) shows the experimental re-sults of KanseiGenie. Figure 7 shows the results of Motelab.Figure 8 shows the results of Tutornet. Figure 9 shows theresults of TWIST. Overall, analytical results match experi-mental results in all four testbeds. However, there are gapsbetween analytical and experimental results for all testbeds.There can be two reasons. First, for Result 2, we assumeinfinite topology by which we collect the averaged distribu-tion of link selection. But, with testbed experiments, wetest 4 × 4 grid topology with limited possible path length.Good long links, bad medium links, and good short linkscombination contributes to the specific experimental resultsof KanseiGenie and Motelab. Second, we consider interfer-ence from external 802.15.4 traffic, or from 802.11 traffic asthe source of variance of link selection, especially for lowerpower cases.To validate how Result 1 (calculates the expected perfor-

mance) performs in the two-dimensional topology, we presentthe analytical results of the end-to-end delay (the total num-ber of transmissions) predicted by Result 1 and comparedthem with the experimental results in figure 10. Overall, an-alytical results match experimental results in all four testbeds.The analytical and the experimental studies on the progressto destination metric are not presented due to space limita-tion.

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5 6Cum

ulat

ive

Usa

ge W

eigh

ts

Progress to Destination

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5 6Cum

ulat

ive

Usa

ge W

eigh

ts

Progress to Destination

Figure 6: Cumulative Usage Weights at KanseiGe-nie. (1) Analytical, (2) Experimental. TransmissionPower = (+:0, ×:-5, �:-10, �:-15) dBm

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5 6Cum

ulat

ive

Usa

ge W

eigh

ts

Progress to Destination

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5 6Cum

ulat

ive

Usa

ge W

eigh

ts

Progress to Destination

Figure 7: Cumulative Usage Weights at Motelab.(1) Analytical, (2) Experimental. TransmissionPower = (+:0, ×:-5, �:-10, �:-15) dBm

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5 6Cum

ulat

ive

Usa

ge W

eigh

ts

Progress to Destination

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5 6Cum

ulat

ive

Usa

ge W

eigh

ts

Progress to Destination

Figure 8: Cumulative Usage Weights at Tutor-net. (1) Analytical, (2) Experimental. TransmissionPower = (+:-5, ×:-7, �:-10, �:-15) dBm

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5 6Cum

ulat

ive

Usa

ge W

eigh

ts

Progress to Destination

0

0.2

0.4

0.6

0.8

1

0 1 2 3 4 5 6Cum

ulat

ive

Usa

ge W

eigh

ts

Progress to Destination

Figure 9: Cumulative Usage Weights at TWIST. (1)Analytical, (2) Experimental. Transmission Power= (+:-5, ×:-10, �:-15, �:-25) dBm

According to figure 10, we can find a set of transmissionpowers which achieve comparable performance (the numberof transmissions) in four testbeds. Table 5 shows exam-ples of performance matching across four testbeds. Overall,experimental results show a little more transmissions thananalytically predicted. Because, the last hop to destinationshould be shorter than other hops (edge effect), we can con-sider this reasonable.

399

0 1 2 3 4 5 6 7 8

-16 -14 -12 -10 -8 -6 -4 -2 0

Num

ber o

f Tra

nsm

issi

ons

Tx. Power (dBm)

KM

TUTW

0 1 2 3 4 5 6 7 8

-16 -14 -12 -10 -8 -6 -4 -2 0

Num

ber o

f Tra

nsm

issi

ons

Tx. Power (dBm)

KM

TUTW

Figure 10: Number of Transmissions. (1) Analyti-cal, (2) Experimental. K: KanseiGenie, M: Motelab,TU: Tutornet, TW:TWIST

KanseiGenie Motelab Tutornet TWIST

Tx. Power -7dBm -10dBm -7dBm -10dBm

Analytical 1.55 1.61 1.68 1.64

Experimental 2.33 2.09 2.43 2.04

Transmissions -10dBm -15dBm -10dBm -15dBm

Analytical 2.25 2.26 2.00 2.49

Experimental 3.10 3.17 3.20 3.06

Table 5: Total Ttransmissions matching Examples

5. CONCLUSIONS AND FUTURE WORKIn this paper, we prosented a technique for achieving re-

peatable performance across different networks deployed indiverse set of RF environments. Our technique accommo-dates network pairs whose signal propagation characteris-tics and inter-node spacings may be different. The methodis executed in three steps: First, the link usage distribu-tion is calculated analytically/ Second, conditional momentsof the link quality are calculated for each link chosen bythe forwarding protocol. Third, link usage and link qual-ity statistics are aggregated to compute expected protocolperformance using a user-specifed metric for each networkand to select the optimal transmission power adjustment tohave same expected protocol performance across the net-works. We provide a multi testbed case study to validatethe proposed method.It is important to acknowledge the limitations of our tool,

as presented. The tool targets consistency to a single cho-sen performance metric. Our experiments however indicatethat this limitation is not particularly restrictive in prac-tice, since attempting to achieve optimal consistency withrespect to one metric seems to result in achieving good con-sistency with respect to other performance metrics as well.Moreover, the tool requires the forwarding protocol to bespecified in order to calculate the link usage probabilities.Alternatively, link usage and conditional link SNR momentscan be computed empirically from in situ observations ofpacket source-destination pair.In the future, we will study how to redress the limita-

tions of the link usage spectrum, focusing in particular onproviding a continuous version of the concept which allowsnetwork nodes to be placed at random points in a geomet-ric space. We will explore predictable performance in thetarget networks, using knobs other than the transmissionpower control and taking into account metrics other thanthose related to the forwarding link selection alone. We willalso study the variability of protocol behavior in one net-work and account for the preservation of that variabilityin the target network, over and above the average behav-

ior that we have focused on in this work. We will studyhow to extend our technique to adopt more user-specifiedperformance indexes (e.g. goodput, energy efficiency, delay,etc.), and reflect more complex RF environment factors (e.g.macro and micro fading effects).

6. REFERENCES[1] T. Kwon, E. Ertin, and A. Arora, “Reproducing

consistent wireless protocol performance acorssenvironments,”Adhocnets, 2010.

[2] KanseiGenie Testbed.http://kansei.cse.ohio-state.edu/KanseiGenie/.

[3] Motelab Testbed. http://motelab.eecs.harvard.edu/.

[4] Tutornet Testbed.http://enl.usc.edu/projects/tutornet/.

[5] TWIST Testbed. http://www.twist.tu-berlin.de.

[6] V. Naik, F. Ertin, H. Zhang, and A. Arora, “Wirelesstestbed bonsai,” 2nd International Workshop onWireless Network Measurement, 2006.

[7] C.-F. Chiasserini and M. Garetto, “Modeling theperformance of wireless sensor networks,” in In IEEEInfocom, 2004.

[8] K. Seada, M. Zuniga, A. Helmy, andB. Krishnamachri, “Energy-efficient forwardingstrategies for geographic routing in lossy wirelesssensor networks,” Sensys, 2004.

[9] M. Zuniga, K. Seada, B. Krishnamachri, andA. Helmy, “Efficient geographic routing over lossy inwireless sensor networks,”ACM Transactions onSensor Networks, vol. 4, pp. 111–143, 2008.

[10] S. Lin, G. Zhou, K. Whitehouse, Y. Wu, J. A.Stankovic, and T. He, “Towards stable networkperformance in wireless sensor networks,” inProceedings of the 2009 30th IEEE Real-Time SystemsSymposium, RTSS ’09, (Washington, DC, USA),pp. 227–237, IEEE Computer Society, 2009.

[11] S. Lanzisera and K. Pister, “Theoretical and practicallimits to sensitivity in ieee 802.15.4 receivers,” IEEEInternational Conference on Electronics, Circuits andSystems, 2007.

[12] TEP 123. http://www.tinyos.net/tinyos-2.x/doc/html/tep123.html.

[13] E. Miluzzo, X. Zheng, K. Fodor, and A. Campbell,“Radio characterization of 802.15.4 and its impact onthe design of mobile sensor networks,” EWSN, 2008.

400