Performance Comparison of Routing Protocols for Cognitive Radio Networks Li Sun, Student Member, IEEE, Wei Zheng, Naveen Rawat, Vikramsinh Sawant, and Dimitrios Koutsonikolas, Member, IEEE Abstract—Cognitive radio networks (CRNs) have emerged as a promising solution to the ever-growing demand for additional spectrum resources and more efficient spectrum utilization. A large number of routing protocols for CRNs have been proposed recently, each based on different design goals, and evaluated in different scenarios, under different assumptions. However, little is known about the relative performance of all these protocols, let alone the tradeoffs among their different design goals. In this paper, we conduct the first detailed, empirical performance comparison of three representative routing protocols for CRNs, under the same realistic set of assumptions. Our extensive simulation study shows that the performance of routing protocols in CRNs is affected by a number of factors, in addition to PU activity, some of which have been largely ignored by the majority of previous works. We find that different protocols perform well under different scenarios, and investigate the causes of the observed performance. Furthermore, we present a generic software architecture for the experimental evaluation of CRN routing protocols on a testbed based on the USRP2 platform, and compare the performance of two protocols on a 6 node testbed. The testbed results confirm the findings of our simulation study. Index Terms—Cognitive radio networks, performance comparison, simulation, testbed Ç 1 INTRODUCTION T HE continuously increasing number of WiFi devices has resulted in growing congestion in the crowded Indus- trial, Scientific, and Medical (ISM) bands, putting a potential limit on the evolution of WiFi networking. On the other hand, some licensed bands, e.g., TV broadcast frequencies, remain largely underutilized. In order to satisfy the ever- growing public demand for additional spectrum resources, in November 2008 the FCC issued a ruling permitting unli- censed users (secondary users, SUs) to operate in the so- called white spaces, i.e., unused portions of the TV broad- cast frequency band, as long as they do not interfere with licensed users (primary users, PUs). This ruling marks the arrival of cognitive radio networks (CRNs). In CRNs, SUs have the ability to sense a wide spectrum range, dynamically identify currently unoccupied spectrum blocks, and choose the best available block to transmit, ensuring non-interfering coexistence with PUs [1]. While research on CRNs was initially focused on PHY/MAC layer issues (e.g., [2], [3], [4], [5]), soon the research community realized the great potential of multihop CRNs. By exploiting the unoccupied frequency resources, the cognitive radio technology is expected to largely increase the capacity of multihop wireless networks [6]. However, the unique characteristics of the white spaces, i.e., spatial variation, spectrum fragmentation, and temporal variation [7], make multihop CRNs very different from mul- tihop networks in the ISM bands. While in traditional wire- less mesh networks (WMNs) the main task of a routing protocol is to discover routes of high quality links, in multi- hop CRNs the main task changes to ensuring radio resour- ces for SU transmissions while guaranteeing the service for all ongoing PU communications [8]. To fulfill this task, rout- ing in CRNs has to address a number of challenges, includ- ing adapting to dynamic changes of spectrum availability, the heterogeneity of resources such as the availability of dif- ferent channels and radios on the same node, and synchro- nization between nodes on different channels [9]. Therefore, designing routing protocols for CRNs is a more challenging task than for networks in the ISM bands. Recently, numerous routing protocols for CRNs have been proposed (e.g., [6], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22]). Besides the main goal of protecting PU transmissions, each protocol is proposed based on different design goals, e.g., maximizing spectrum opportunities, maximizing available bandwidth, minimiz- ing hopcount, minimizing end-to-end delay, etc. The per- formance of each protocol is evaluated with respect to its specific design goals and sometimes compared against a baseline protocol (e.g., random routing). Moreover, each protocol is evaluated using a different evaluation method- ology—different assumptions (e.g., about PU activity), set- tings, and scenarios, tailored to its specific design goals. Although each methodology offers a deeper understanding of a specific protocol, little is known about the relative per- formance of all these protocols, let alone the tradeoffs among their different design goals. While extensive perfor- mance comparisons have been conducted for multihop routing protocols in the ISM band (e.g., for MANETs [23], L. Sun, W. Zheng, and D. Koutsonikolas are with the Computer Science and Engineering Department, University at Buffalo, The State University of New York, Buffalo, NY 14260-2500. E-mail: {lsun3, wzheng4, dimitrio}@buffalo.edu. N. Rawat is with Qualcomm Atheros, 1700 Technology Drive, San Jose, CA 95110. E-mail: [email protected]. V. Sawant is with Dow Jones & Co., 4300 US Highway 1 Monmouth Junction, NJ 08852. E-mail: [email protected]. Manuscript received 15 Sept. 2013; revised 8 July 2014; accepted 2 Aug. 2014. Date of publication 10 Aug. 2014; date of current version 1 May 2015. For information on obtaining reprints of this article, please send e-mail to: [email protected], and reference the Digital Object Identifier below. Digital Object Identifier no. 10.1109/TMC.2014.2346782 1272 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 14, NO. 6, JUNE 2015 1536-1233 ß 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Transcript
Performance Comparison of Routing Protocolsfor Cognitive Radio Networks
Li Sun, Student Member, IEEE, Wei Zheng, Naveen Rawat, Vikramsinh Sawant, and
Dimitrios Koutsonikolas,Member, IEEE
Abstract—Cognitive radio networks (CRNs) have emerged as a promising solution to the ever-growing demand for additional spectrum
resources andmore efficient spectrum utilization. A large number of routing protocols for CRNs have been proposed recently, each
based on different design goals, and evaluated in different scenarios, under different assumptions. However, little is known about the
relative performance of all these protocols, let alone the tradeoffs among their different design goals. In this paper, we conduct the first
detailed, empirical performance comparison of three representative routing protocols for CRNs, under the same realistic set of
assumptions. Our extensive simulation study shows that the performance of routing protocols in CRNs is affected by a number of factors,
in addition to PU activity, some of which have been largely ignored by themajority of previous works.We find that different protocols
performwell under different scenarios, and investigate the causes of the observed performance. Furthermore, we present a generic
software architecture for the experimental evaluation of CRN routing protocols on a testbed based on the USRP2 platform, and compare
the performance of two protocols on a 6 node testbed. The testbed results confirm the findings of our simulation study.
Index Terms—Cognitive radio networks, performance comparison, simulation, testbed
Ç
1 INTRODUCTION
THE continuously increasing number of WiFi devices hasresulted in growing congestion in the crowded Indus-
trial, Scientific, and Medical (ISM) bands, putting a potentiallimit on the evolution of WiFi networking. On the otherhand, some licensed bands, e.g., TV broadcast frequencies,remain largely underutilized. In order to satisfy the ever-growing public demand for additional spectrum resources,in November 2008 the FCC issued a ruling permitting unli-censed users (secondary users, SUs) to operate in the so-called white spaces, i.e., unused portions of the TV broad-cast frequency band, as long as they do not interfere withlicensed users (primary users, PUs). This ruling marks thearrival of cognitive radio networks (CRNs).
In CRNs, SUs have the ability to sense a wide spectrumrange, dynamically identify currently unoccupied spectrumblocks, and choose the best available block to transmit,ensuring non-interfering coexistence with PUs [1]. Whileresearch on CRNs was initially focused on PHY/MAC layerissues (e.g., [2], [3], [4], [5]), soon the research communityrealized the great potential of multihop CRNs. By exploitingthe unoccupied frequency resources, the cognitive radiotechnology is expected to largely increase the capacity ofmultihop wireless networks [6].
However, the unique characteristics of the white spaces,i.e., spatial variation, spectrum fragmentation, and temporalvariation [7], make multihop CRNs very different frommul-tihop networks in the ISM bands. While in traditional wire-less mesh networks (WMNs) the main task of a routingprotocol is to discover routes of high quality links, in multi-hop CRNs the main task changes to ensuring radio resour-ces for SU transmissions while guaranteeing the service forall ongoing PU communications [8]. To fulfill this task, rout-ing in CRNs has to address a number of challenges, includ-ing adapting to dynamic changes of spectrum availability,the heterogeneity of resources such as the availability of dif-ferent channels and radios on the same node, and synchro-nization between nodes on different channels [9]. Therefore,designing routing protocols for CRNs is a more challengingtask than for networks in the ISM bands.
Recently, numerous routing protocols for CRNs havebeen proposed (e.g., [6], [10], [11], [12], [13], [14], [15], [16],[17], [18], [19], [20], [21], [22]). Besides the main goal ofprotecting PU transmissions, each protocol is proposedbased on different design goals, e.g., maximizing spectrumopportunities, maximizing available bandwidth, minimiz-ing hopcount, minimizing end-to-end delay, etc. The per-formance of each protocol is evaluated with respect to itsspecific design goals and sometimes compared against abaseline protocol (e.g., random routing). Moreover, eachprotocol is evaluated using a different evaluation method-ology—different assumptions (e.g., about PU activity), set-tings, and scenarios, tailored to its specific design goals.Although each methodology offers a deeper understandingof a specific protocol, little is known about the relative per-formance of all these protocols, let alone the tradeoffsamong their different design goals. While extensive perfor-mance comparisons have been conducted for multihoprouting protocols in the ISM band (e.g., for MANETs [23],
� L. Sun, W. Zheng, and D. Koutsonikolas are with the Computer Scienceand Engineering Department, University at Buffalo, The State Universityof New York, Buffalo, NY 14260-2500.E-mail: {lsun3, wzheng4, dimitrio}@buffalo.edu.
� N. Rawat is with Qualcomm Atheros, 1700 Technology Drive, San Jose,CA 95110. E-mail: [email protected].
� V. Sawant is with Dow Jones & Co., 4300 US Highway 1 MonmouthJunction, NJ 08852. E-mail: [email protected].
Manuscript received 15 Sept. 2013; revised 8 July 2014; accepted 2 Aug. 2014.Date of publication 10 Aug. 2014; date of current version 1 May 2015.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference the Digital Object Identifier below.Digital Object Identifier no. 10.1109/TMC.2014.2346782
1272 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 14, NO. 6, JUNE 2015
1536-1233� 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
[24] or WMNs [25]), almost a decade since the first CRNrouting protocol was proposed, there has been (to our bestknowledge) no extensive performance comparison of rout-ing protocols for CRNs.
In this paper, we conduct the first extensive empiricalperformance study of routing protocols for CRNs usingboth a simulator (ns-2) and a testbed based on the USRP2platform [26], under the same realistic set of assumptions:1) SUs have no knowledge about PU activity models andparameters; 2) each SU senses PU activity independentlyand periodically and learns PU activity online; 3) PUs caninterrupt SU communications at any time; 4) the only wayfor two SUs to learn information about each other (e.g.,observation of PU activity in each other’s neighborhood) isthrough communication.
In the simulator, we implement and compare three rep-resentative routing protocols for CRNs—Coolest Path [12],SAMER [10], and CRP [13], each with different designobjectives. Coolest Path aims to find the path with thehighest spectrum availability, which results in path stabil-ity. SAMER tries to find the path with the highest through-put by taking into account both PU and SU activity, aswell as link quality. CRP is designed to either find a pathwith minimum end-to-end delay and satisfactory PU pro-tection or offer the best protection to PU receivers at thecost of some performance degradation for SUs. Further-more, each protocol takes a different approach with respectto four basic building blocks of a CRN protocol, including1) how to characterize spectrum opportunities observedlocally, 2) how to characterize spectrum opportunitiesbetween neighboring SUs, 3) how to define a link metricbased on spectrum opportunities and 4) how to select arouting path based on the link metric. Our study revealsthe pros and cons of each approach as well as the tradeoffsamong the different design goals in a variety of scenarios.
Our main findings are summarized as follows: 1) Underlow PU activity, path stability is not the only factor thataffects the performance of a CRN protocol; factors consid-ered by traditional WMN routing protocols, such as linkquality and interference among neighboring nodes, shouldalso be taken into account. In such scenarios, SAMER out-performs the other two protocols in terms of both through-put and end-to-end delay. In the presence of multiple flows,SAMER improves the total throughput at the cost of a smallreduction in fairness, as it tries to choose disjoint paths fordifferent flows. 2) Under high PU activity, path stability andpath length become the dominant factors that affect perfor-mance. In such scenarios, Coolest Path with an additivepath metric outperforms the other protocols. 3) When thelink routing metric ignores link quality, an additive pathmetric in general performs better than a bottleneck metric,as it limits the path length. In contrast, when link quality istaken into account, longer paths often yield better perfor-mance; a similar observation has been made for routingmetrics proposed for traditional WMNs, e.g., ETX [27] orETT [28]. 4) It is important to consider neighbor observa-tions in estimating spectrum opportunities, due to spatialvariation in PU activity [7]. CRP often performs poorly,because it estimates spectrum opportunities based only onlocal observation. 5) When channels have different propaga-tion characteristics, reducing path length by choosing long-
propagation channels improves end-to-end delay at the costof lower path stability and throughput.
Furthermore, we develop a generic software architecturefor the experimental evaluation of CRN routing protocolson a testbed based on the USRP2 platform. Our frameworkprovides an implementation of PHY, MAC, and networklayers, which can be used as the basic building blocks forthe implementation of any routing protocol. Basic CRNfunctions such as emulation of PU activity, SU periodicsensing, and channel switching capabilities are also sup-ported. Neighboring SUs use a common control channel forexchanging control messages in a distributed way. Based onthis architecture, we implement and compare Coolest Pathand SAMER on a six-node testbed. Our testbed results agreewith the findings of our simulation study. In spite of thesmall size of the testbed, we note that this is the first (to ourbest knowledge) testbed-based performance comparison oftwo CRN routing protocols.
The rest of the paper is organized as follows. Section 2briefly reviews the three CRN protocols we study in thispaper and discusses related work. Section 3 describes oursimulation methodology. In Section 4, we present our simu-lation study. In Section 5, we describe the testbed architec-ture and the experimental evaluation. Finally, Section 6concludes the paper.
2 RELATED WORK AND BACKGROUND
In this section, we provide an overview of the related workand we briefly describe the three routing protocols we con-sider in our study.
2.1 Related Work
In recent years, numerous routing protocols for CRNs havebeen proposed with different design goals, e.g., maximizingthroughput [14], [15], [10], [16], [17], [18], minimizing delay[11], [19], [6], [13], maximizing route stability [20], [12], min-imizing route recovery/maintenance cost [21], [22], etc.Many of those protocols [14], [15], [20], [11], [16], [19], [17]assume static channel availability and do not include PUdynamics in their routing metrics. Consequently, such pro-tocols are similar to those proposed for multi-channelWMNs and cannot deal with temporal variations of spec-trum availability in CRNs. Among works which take PUdynamics into account, some focus on analytical studies,e.g., [22] and [18], and some others propose protocols rely-ing on transmission power adaptation, e.g., [18] and [6]. Aperformance comparison of these types of protocols is leftas future work. Among the remaining protocols, we chosethe ones in [12], [10], and [13] for our study, as [21] only con-siders mesh networks arranged in a tree topology.
The majority of routing protocols for CRNs are only com-pared against protocols which do not take PU dynamics intoaccount, e.g., [10], [21], [12], [13]. One exception is [6], inwhich the authors compare the proposed protocol againstSAMER. In their evaluation, the authors do not implementa sensing functionality on the SUs to learn PU activityonline, in a distributed way, but instead they assume eachSU has complete a priori knowledge of the model andparameters of PU activity. In [30], the authors conduct acomparison study of single-path and multi-path AODV
SUN ET AL.: PERFORMANCE COMPARISON OF ROUTING PROTOCOLS FOR COGNITIVE RADIO NETWORKS 1273
with three routing metrics—ETX, ETT, and hop count—in amultihop CRN. However, neither AODV nor any of thethree routing metrics are designed for CRNs. To our bestknowledge, our work is the first extensive empirical perfor-mance comparison study of routing protocols in CRNs.
Most of the existing routing protocols for CRNs haveonly been evaluated in simulators, primarily due to the dif-ficulty to build a CRN testbed. Although a number of CRplatforms has become available in the past few years (see[36] for a survey), the majority of CRN protocols that havebeen evaluated on testbeds are MAC/PHY layer protocols(e.g., [7], [34]).
A notable exception is Coolest Path [12] which was eval-uated in a six-node USRP-based testbed. The authors in [12]only compare Coolest Path against random routing andthey do not provide details about the testbed architecture,e.g., about the MAC/Network layer or the implementationof the common control channel. Furthermore, in theirtestbed evaluation, the authors use the route switch ratio(which is proportional to the number of route breaks) as theperformance metric. In our study, we show that a lowerswitch ratio (number of route breaks) does not always resultin higher throughput.
The authors in [34] and [35] build a small testbed of 3USRPs for the evaluation of their proposed routing proto-cols. The testbed has some similarities to ours, e.g., the com-mon control channel is implemented as an Ethernetinterface. However, their evaluation included only single-hop experiments. In contrast, in our testbed, we evaluatethe performance of two routing protocols in multihop topol-ogies with more USRPs.
2.2 Background
Coolest Path [12] tries to find the most stable path, i.e., thepath with the most balanced and/or lowest spectrum uti-lization by PUs. In Coolest Path, a channel’s temperaturefor an SU link is defined as the fraction of time duringwhich the channel is unavailable due to PU activity in theneighborhood of any of the two SUs. The link’s tempera-ture is then defined as the minimum channel temperatureamong all available channels between the two SUs. Cool-est Path provides three different definitions of the pathtemperature based on the link temperature: (i) accumulatedtemperature, i.e., the sum of the link temperatures alongthe path, (ii) highest temperature, i.e., the maximum linktemperature among the links along the path, and(iii) mixed temperature—a combination of the first two.The protocol selects the path with the minimum path tem-perature. In [12], the performance of mixed temperaturewas always found to lie between the performance of theother two path metrics. For this reason, we do not con-sider mixed temperature in our study.
SAMER [10] tries to find a high-throughput path byopportunistically utilizing high-throughput links while stillguaranteeing a path’s long-term stability. To quantify chan-nel availability, SAMER considers both PU and SU activity.Each SU estimates the fraction of time during which a chan-nel can be used, i.e., it is not used by any PU and any otherSU. Since two neighboring nodes may estimate differentchannel availabilities, the channel availability for a link isgiven by the smallest of the two values. SAMER’s link metric
is based on ETT [28], one of themost popular routingmetricsfor traditional WMNs. For each channel, SAMER estimatesthe expected throughput as the product of channel availabil-ity, link bandwidth, and loss rate. The link metric is thendefined as the sum of throughput values of all availablechannels. Hence, different from Coolest Path’s link tempera-ture, which reflects only a link’s stability, the link metric inSAMER reflects both link stability (channel availability) andlink quality (bandwidth, loss rate). The path metric is theminimum throughput among all links along a path, i.e., abottleneck metric. Since a bottleneck metric may yield verylong paths, [10] uses a heuristic which selects the path withthe smallest possible number of hops that has a cost lowerthan or equal to amaximum costCmax.
CRP [13] considers two different routing classes thatoffer different levels of protection to PUs. Class I aims tominimize the end-to-end delay while still providing satis-factory protection to PUs. On the other hand, Class IIallows a level of performance degradation and prioritizesPU protection by selecting as relays SUs that are far fromPU receivers. Since in this study we focus on perfor-mance, we only consider Class I routes. In CRP, when anSU receives a route request, it selects a rebroadcast delayby calculating a cost function based only on local infor-mation. The cost function considers the SU’s estimates ofchannel availabilities, variance of intensity of PU activity,etc. An SU with a lower cost will rebroadcast the routerequest earlier. When the destination SU receives a routerequest, it simply sends a route reply back along the pathover which it received the route request, without per-forming any local computation. Based on this cost-delaymapping, CRP can be easily implemented via minor mod-ifications to AODV [29].
Table 1 summarizes the differences among the three pro-tocols in the estimation of (i) channel availability for a nodeor a link, (ii) link metric, and (iii) path metric.
3 SIMULATION METHODOLOGY
For our simulation study, we adopted the ns-2 extendedframework proposed in [30], which implements all neces-sary components for SUs in a CRN [31]: a spectrum sensingblock (for detecting PU activity), a spectrum mobility block(for performing spectrum handoff after detecting a PU onthe current channel), a spectrum decision block (for channelselection), and a spectrum sharing block (for allowing SUsto share the spectrum and avoid collisions through carriersensing). Similar to in [13], [30], each SU is equipped withone receiving interface for receiving data packets and sens-ing the spectrum and one transmitting interface for sendingdata packets. There is also a third interface fixed on the con-trol channel and used only for transmitting/receiving con-trol packets, e.g., route requests/replies and channelswitching notifications.
In our simulations, we use a 1-sec sensing-transmissioncycle; SUs sense the spectrum in the first 0.1 sec (sensingperiod) and use the remaining 0.9 sec (data transmissionperiod) to send/receive data. The sensing periods on allSUs are synchronized according to the 802.22 standard [32].SUs switch to a new channel rather than waiting on the pre-vious one when they detect PU activity on the current
1274 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 14, NO. 6, JUNE 2015
channel. Although an SU is not able to detect PU activityduring the data transmission period, our choice of the sens-ing cycle parameters guarantees that an SU will vacate achannel used by a PU in less that 2.0 sec, which still meetsthe requirement of the 802.22 standard [32].
In [30], PU activity follows an exponential ON-OFFmodel proposed in [2]. We use the same model in our study.The ON state represents the time interval during which achannel is occupied by a PU and the OFF state representsthe interval during which a PU is idle and the channel canbe used by SUs. Each PU switches between the ON and theOFF state. Although SUs cannot detect PU activity duringthe data transmission period, the simulator [30] models theimpact of PU activity on SU transmissions by assuming a20 percent packet loss probability (due to collision) if a PUis active during an SU transmission.
At the PHY layer, we assume a spectrum band of 11orthogonal channels with the propagation characteristics of2.4 GHz. Ten of these channels can be used for data trans-missions and one is used as the common control channel.Each interface can be tuned to one channel at a time. Thebandwidth of each channel is 6 MHz, which is the same as aTV channel in the UHF band. To simulate different channelqualities, we assume that the packet loss ratio for everychannel follows a uniform distribution between 0 and amaximum loss ratio. The default maximum packet loss ratiois set to 0.2.
At the MAC layer, we use 802.11b and disable RTS/CTS.To reflect a channel width of 6 MHz (instead of 20 MHzused in 802.11), we scale down the 802.11b data rates by afactor of 6 MHz/20 MHz. In our simulations, SUs use thehighest data rate of 3.3 Mbps (11 Mbps in 802.11b) to trans-mit data packets and the basic data rate of 0.3 Mbps(1 Mbps in 802.11b) to broadcast control packets.
At the network layer, we modified AODV to support thethree CRN routing protocols. When a route discovery is ini-tiated, a RREQ packet is created at the source and is floodedtowards the destination. When a node broadcasts a RREQ,it appends the link metric for the link through which itreceived the RREQ. The destination receives a number ofRREQs over different paths, chooses the least cost pathaccording to the path metric used by the routing protocol,and sends a RREP packet back to the source along the cho-sen path. Neighboring SUs on the chosen path select thebest channel among all available ones according to the linkmetric. When the selected channel can no longer be usedbecause of PU activity, i.e, the link breaks, the sender-receiverpair tries to repair the link locally by selecting another
channel, which is the best among the currently availablechannels according to the link metric. When no channel isavailable, the link cannot be used anymore, i.e., the routebreaks, and a RERR packet is forwarded along the routingpath. When the RERR packet arrives at the source node, anew route discovery is initiated by the source node, whichbuffers packets during this process.
4 SIMULATION STUDY
In this section, we first introduce our simulation setup andthen discuss the simulation results.
4.1 Simulation Setup
We use the topology shown in Fig. 1, which is similar to theones used in [6] and [13]. A square region of side 1,200 m isdivided into nine square cells of side 400 m. There are ninePU locations in the centers of the cells. In each location,there are 10 PUs, operating on the 10 channels which can beused for data transmissions; there is no PU operating on thecommon control channel. Each PU has an interference rangeof 250 m. 49 SUs are placed on a grid; the distance betweenany two neighboring SUs is 160 m. Each SU has a maximumtransmission range of 250 m on each channel. We use SU0in cell 1 as the source node and SU9 in cell 3 as the destina-tion node, unless otherwise stated.
Each simulation runs for 900 sec, during which PUs maybecome active at any time. In the first 600 sec, SUs only
Fig. 1. Simulation topology.
TABLE 1Qualitative Comparison among the Three Protocols Considered in This Study
Protocol Node Channel Availability Link Channel Availability Link Metric Path Metric
Cost function reflectingdelay or protectionto PU receivers
Accumulatedlink metric values
SUN ET AL.: PERFORMANCE COMPARISON OF ROUTING PROTOCOLS FOR COGNITIVE RADIO NETWORKS 1275
sense the spectrum (during the sensing periods) to learn thestatistics of PU activity in their neighborhood. The sourcenode starts transmitting data packets at the 600th secondand the data transmission period lasts for 300 sec, duringwhich SUs keep sensing the spectrum and updating theirobservations of PU activity. Each source sends CBR datatraffic over UDP at a rate of 3.3 Mbps with a packet size of1,500 Bytes.
We compare the performance of the three routing proto-cols based on four metrics: throughput, number of brokenlinks, number of broken routes, and average end-to-enddelay of successfully delivered packets at the applicationlayer. We chose throughput as our primary performancemetric following the trend in the evaluation of routing pro-tocols for traditional mesh networks [27], [28]. For scenarioswith multiple flows (Section 4.6), we also evaluate fairnessusing Jain’s Fairness Index [33].
For Coolest Path, we consider two versions: Coolest Pathwith Accumulated Temperature (CP-AT) and with HighestTemperature (CP-HT). For SAMER, we found that it isimpossible to select a Cmax value that works well for all sce-narios. Hence, instead of using a path cost threshold to con-trol path length, we use a hop count threshold: a � SPC,where SPC is the hopcount of the shortest path between asource-destination pair and a is a parameter controlling theactual path length. We found that a ¼ 2 achieves a betterbalance between long term stability and short term perfor-mance. Finally, for both CP-HT and SAMER, which use bot-tleneck path metrics, we give preference to shorter pathsamong paths with the same path metric value.
4.2 Baseline Scenario
We first compare the throughput of the four routing proto-cols in a baseline scenario, in which we assume all PUs havethe same average ON and OFF times. For each combinationof average ON/OFF times, we repeat the simulation20 times, using each time a different seed to generate PUactivity, i.e., ON/OFF intervals following an exponentialdistribution.
In Fig. 2a, we fix the average PU OFF time at 6 sec andvary the average ON time from 3-10 sec. Each point corre-sponds to the average throughput over 20 simulation runsand the error bars correspond to the standard deviations.We observe that the throughput of all four routing protocolsdrops when the intensity of PU activity increases. In thisscenario, we find CP-AT always performs better than CP-HT while the performance of CRP lies between CP-AT andCP-HT. Interestingly, SAMER outperforms the other rout-ing protocols when the average ON time is smaller than6 sec and exhibits the worst performance when the averageON time is larger than 6 sec.
In Fig. 2b, we fix the average PU ON time at 6 sec andincrease the average OFF time from 3-10 sec. We observethat the relative performance of the four protocols is thesame as in Fig. 2a. Again, CP-AT outperforms CRP and CP-HT while SAMER’s performance is the best when the inten-sity of PU activity is low and the worst when the intensityof PU activity is high.
Since the performance trend is the same when we varyeither the OFF or the ON time, for simplicity, in the rest ofthe paper we always fix the average PU ON time at 6 secand vary the average PU OFF time. In Sections 4.3, 4.4, westudy the performance of the four protocols in more com-plex scenarios with respect to PU activity and investigatethe causes of the observed performance. In Secion 4.5, westudy the impact of the channel loss ratio. In Sections 4.6and 4.7, we study the performance in the presence of multi-ple flows and with different source-destination pairs.Finally, in Section 4.8, we study the impact of varying spec-trum propagation characteristics.
4.3 Localized PU Activity
We now compare the four protocols in a more realistic sce-nario where PU activity varies in different locations. In [7],the authors point out that rural and suburban regionsexhibit a much lower degree of spectrum fragmentationand more contiguous spectrum than urban areas. To simu-late this scenario, we use different average OFF times forPUs in different cells. In each cell, all PUs are assigned thesame average OFF time, chosen uniformly from the interval2-11 sec. We use 200 different seeds to select average OFFtimes and generate PU activity.
Throughput comparison. Fig. 3a plots the Cumulative Dis-tribution Function (CDF) of the 200 throughput values foreach protocol. SAMER performs the best in 80 percent ofthe scenarios and CRP performs the worst among the fourprotocols while the performance of CP-HT is worse thanbut very close to the performance of CP-AT. In the mediancase, SAMER outperforms CP-AT by 17.29 percent and CRPby 55.91 percent.
Path lengths. Fig. 3b plots a scatterplot of the throughputagainst the average routing path length for each of the 200
Fig. 2. Baseline throughput comparison. For better clarity, the datapoints for CP-AT, CP-HT, and CRP are shifted horizontally.
1276 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 14, NO. 6, JUNE 2015
simulation runs. We observe that, in general, SAMER choo-ses longer routes than the other protocols. There are tworeasons for this behavior. First, SAMER’s link metric consid-ers both spectrum opportunities and link qualities. Hence,the protocol often prefers longer paths consisting of higherquality links, similar to link quality-based routing metricsfor traditional WMNs, e.g., ETX or ETT. Second, SAMER’spath routing metric is a bottleneck metric; hence, the routeselection does not prefer shorter paths.
CP-HT’s path metric is also a bottleneck metric, but itslink metric considers only PU activity. As a result, in mostcases its path length is shorter than SAMER’s. On the otherhand, CP-AT and CRP tend to choose shorter paths becauseof their additive path metrics. However, CRP sets a lowerlimit on spectrum availabilities and an upper limit on spec-trum availability variance, and, in some cases, it prefers alonger path, when nodes on the shortest path do not satisfythese constraints.
In spite of choosing longer paths, SAMER still achievesthe highest throughput among the four protocols in mostcases. This is because SAMER’s link metric also takes con-tention among SUs into account, i.e., it tries to avoid assign-ing the same channel to two nodes in the same interferencerange. More importantly, SAMER is also able to find morestable paths. SAMER considers the potential throughputvalues over all possible channels and avoids SUs with onlyone good channel. As a result, the protocol selects relays incells with lower PU activity gaining advantage in scenarioswhere the intensity of PU activity varies per cell. Thisadvantage is lost when all cells exhibit high PU activity(Figs. 2a and 2b in the baseline scenario).
Broken links/routes. Figs. 3c and 3d plot the CDFs of thenumber of broken links and broken routes, respectively, foreach protocol. We observe that CP-AT and CP-HT experi-ence the smallest number of broken links and broken routes,
respectively; this demonstrates the protocol’s effectivenessin finding stable paths, and is consistent with the protocol’sprimary design goal. Although CP-HT experiences morebroken links than CP-AT, the number of broken routes islower. This is because CP-HT avoids locations with high PUactivity more aggressively without considering pathlengths; although CP-HT experiences more broken links(because of longer paths), it has a better chance to find anew channel to switch to during link breaks which avoidsan end-to-end route discovery.
SAMER experiences the largest number of broken links(31 percent more than CP-AT in the median case), however,most of the link breaks do not result in route breaks; its bro-ken routes are even fewer than CP-AT’s (median case com-parison: 35 versus 41). SAMER prefers links with higheravailability on all channels and is often able to switch chan-nel successfully during link breaks. Furthermore, similar toCP-HT, SAMER also benefits from this scenario with a bot-tleneck path metric by avoiding locations under higher PUactivity. However, a large number of route breaks affectsSAMER more severely than CP-AT/HT because ofSAMER’s longer paths. We found that most of the scenariosfor which SAMER’s throughput is lower than the 20th per-centile are characterized by very long paths and very largenumber of broken routes. This explains why SAMER’s 20th-percentile throughput is lower than CP-AT’s or CP-HT’s.
CRP experiences the largest number of broken brokenroutes although it has fewer broken links than SAMER. Thisis due to the fact that CRP estimates statistics of PU activityusing only local observations and ignores PU activity on theother end of the link. Although CRP sets a threshold onchannel availability locally, it is not guaranteed that a linkcan be indeed utilized with a probability that satisfies thatthreshold when neighboring SUs are impacted by PUs indifferent locations. Consequently, the protocol suffers from
Fig. 3. Localized PU activity.
SUN ET AL.: PERFORMANCE COMPARISON OF ROUTING PROTOCOLS FOR COGNITIVE RADIO NETWORKS 1277
a large number of broken routes, which result in lowthroughput, as shown in Fig. 3a.
End-to-end delay. Fig. 3e plots the CDFs of the end-to-enddelay for each protocol. We note that we used a large buffersize at the link layer in the simulator, as our main focus wason throughput and not on delay; this resulted in long end-to-end delays. We observe that SAMER achieves the bestperformance in most cases, simultaneously optimizingthroughput and delay, because it is the only protocol thattakes SU contention into account. In particular, it even out-performs CRP, although CRP is designed to minimize end-to-end-delay. Among protocols which do not take SU con-tention into account, the path length determines the end-to-end delay; CP-HT experiences the longest delays due to itslonger paths compared to CRP and CP-AT.
4.4 Random PU Activity
In this section, we simulate a scenario in which the averageOFF time for each PU is chosen uniformly from the interval2-11 sec, independent of its location. Compared to the sce-nario in 4.3, this scenario is characterized by more diversityin terms of PU activity. Similar to 4.3, we use 200 differentseeds to select average OFF times and generate PU activity.
Throughput comparison. Fig. 4a plots the throughput CDFsfor each protocol and Fig. 4b plots the throughput againstthe average path length for each run. We observe that inthis scenario CP-AT clearly outperforms CP-HT. The reasonis the difference in the path lengths. The median path lengthfor CP-HT is 7.74 hops while CP-AT chooses the shortestpath (6 hops) in almost all cases. In contrast, in the scenarioof Section 4.3, CP-HT chose much shorter paths (Fig. 3b).We also observe that SAMER and CP-AT outperform CRPand CP-HT. However, in contrast to the localized PU activ-ity scenario in 4.3, there is no clear winner in this scenario;the median throughput is almost the same for both proto-cols. CP-AT outperforms SAMER in half of the simulation
runs (those yielding throughputs lower than 0.7 Mbps) andSAMER outperforms CP-AT in the other half.
Broken links/routes. We plot the CDFs of the number ofbroken links in Fig. 4c, and the CDFs of the number of bro-ken routes in Fig. 4d. We observe that SAMER again has thelargest number of broken links and CRP has the largestnumber of broken routes, similar to in Section 4.3. CP-AT isthe most stable protocol, experiencing the smallest numberof both broken links and routes. CP-HT has a much smallernumber of broken routes compared to CRP but it appears tobe more sensitive to broken routes—even a small number ofbroken routes results in low throughput, as we saw inFig. 4a, which is similar to what happened to SAMER in Sec-tion 4.3. The reason is the long routing paths which result inhigh packet loss in case of route breaks, since there are morepackets buffered along a long path.
End-to-end delay. Fig. 4e plots the CDFs of the end-to-enddelay for each protocol. SAMER again performs the bestand CP-HT the worst, similar to in Fig. 3e.
Varying PU activity. To study the performance of SAMERand Coolest Path in more detail, we divide the average PUOFF time into three smaller ranges—2-5 sec, 5-8 sec, and 8-11 sec. For each range, we repeated the simulation with 100different seeds. In Figs. 5a, 5b, and 5c, we plot the through-put CDFs for the three ranges. We observe that the perfor-mance of SAMER drops as the average PU OFF timedecreases - SAMER performs the best in the range of 8-11sec and the worst in the range of 2-5 sec, similar to the base-line scenario in Section 4.2. This also explains the overallresult in Fig. 4a.
Fig. 5d plots the CDF of the number of broken routes forthe range of 2-5 sec. The number of broken routes forSAMER and CP-HT is very high compared to Figs. 3d, 4d—the median numbers are 162 and 165, respectively. More-over, SAMER selects longer paths than CP-HT (themedian path length for the two protocols is 8.86 and 7.86
Fig. 4. Random PU activity—Avg. PU OFF time: 2-11 sec.
1278 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 14, NO. 6, JUNE 2015
hops, respectively). The combined effect of many routebreaks and large path lengths makes SAMER performpoorly under high PU activity. This also explainsSAMER’s poor performance under high PU activity in thebaseline scenario.
4.5 Impact of Channel Loss Ratio
To study the impact of loss ratio on protocol performance,we consider again the baseline scenario and repeat the sim-ulations for maximum packet loss ratio of 0.0, 0.4, and 0.8.The results are plotted in Figs. 6a, 6b, for average PU OFFtime equal to 9 sec and 3 sec, respectively.
In Fig. 6a, we observe that under low PU activity, SAMERachieves the highest throughput among the four protocols.Furthermore, SAMER is the most robust protocol to packetloss. When the maximum packet loss ratio increases from 0.0to 0.8, SAMER’s throughput drops by only 34 percent, whilethe throughput of the remaining three protocols drops byabout 55 percent. When the intensity of PU activity is low,incorporating loss ratio in the link metric of a CRN protocolimproves the protocol’s performance, similar to the case ofroutingmetrics for traditionalWMNs.
On the other hand, Fig. 6b shows that SAMER performsthe worst among the four protocols under high PU activity.
In that case, incorporating loss ratio in the routing metricdoes not help and protocols which ignore loss ratio andchoose routes using PU activity as the only criterion achievebetter performance.
4.6 Multiple Data Flows
To study the performance of each protocol with multipledata flows, we conducted simulations with three and fiveflows using the baseline scenario. In the topology shown inFig. 1, we selected the source-destination pairs SU0-SU9,SU1-SU8, and SU2-SU7, for the simulations with three dataflows. For five data flows, we added two more source-destination pairs—SU3-SU6 and SU4-SU5. We repeated thesimulations with the same 20 seeds used in Section 4.2 foraverage PU OFF time equal to 3 and 9 sec. Fig. 7a plots thetotal throughput with one, three, and five flows, and Fig. 7bplots Jain’s Fairness Index, with three and five flows.
In Fig. 7a, we observe that SAMER achieves the highestthroughput, regardless of the number of flows under lowPU activity (OFF time 9 sec). Moreover, when the numberof flows increases from 1 to 3, the total throughput increaseswith SAMER but drops with the other three protocols.This is because protocols which only consider PU
Fig. 5. Random PU activity—Varying PU OFF time.
Fig. 6. Average throughput as a function of the loss ratio. Fig. 7. Multiple data flows.
SUN ET AL.: PERFORMANCE COMPARISON OF ROUTING PROTOCOLS FOR COGNITIVE RADIO NETWORKS 1279
activity in estimating spectrum availabilities are morelikely to share a large part of the routing path for allthree flows (note that all three source/destination nodesare impacted by PUs in the same cell), resulting in highcontention among SUs. In contrast, SAMER considersinterference from neighboring SUs in estimating spec-trum availability and selects more disjoint paths. InFig. 7b, we observe that the increase in SAMER’s totalthroughput under low PU activity comes at the cost ofreduced fairness compared to the other three protocols;SAMER penalizes some flows by routing them over lon-ger paths in attempt to reduce the amount of SU interfer-ence. In contrast, CP-AT and CRP have the highestfairness, at the cost of reduced throughput.
On the other hand, under high PU activity (OFF time3 sec) SAMER achieves, in general, the lowest perfor-mance in terms of both throughput and fairness, similarto our observations in Sections 4.2 and 4.4. CP-ATachieves the highest throughput followed closely by CRP.Moreover, CP-AT achieves better fairness than CRP, espe-cially as the number of flows increases. CP-HT outper-forms SAMER in terms of throughput in the presence of asingle flow, but the gap diminishes with multiple flows.
Its fairness index though remains higher than SAMER’s,regardless of the number of flows.
4.7 Different Source-Destination Pairs
In this section, we randomly select 50 source-destinationpairs in Fig. 1, with each source and destination impactedby PUs in different locations.1 For each source-destinationpair, we repeat the simulation with 10 different seeds and 3different PU OFF times, namely 9, 6, and 3 sec.
Throughput comparison. Figs. 8a, 8b, and 8c plot thethroughput CDFs of each protocol when the average PUOFF time is 9, 6, and 3 sec, respectively. Similar to inSections 4.2 and 4.4, we observe that SAMER performs thebest under low PU activity and CP-AT performs the bestunder high PU activity. In the median case, SAMER outper-forms CP-AT by 19.92 percent under low PU activity(Fig. 8a) but CP-AT outperforms SAMER by 68.39 percentunder high PU activity (Fig. 8c). Moreover, similar to in Sec-tion 4.2, CP-AT always performs better than CP-HT and theperformance of CRP lies between CP-AT and CP-HT.
Fig. 8. Different source-destination pairs.
1. For source-destination pairs in the same location, all protocolsachieve the same performance.
1280 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 14, NO. 6, JUNE 2015
Path lengths. Figs. 8d and 8g plot the path length CDFsfor each protocol under low and high PU activity, respec-tively. We observe that SAMER and CP-HT choose longerrouting paths than CP-AT and CRP regardless of the inten-sity of PU activity.
We further classify throughput results according to the hopdistance (i.e., the shortest path length) between the source anddestination pairs in Tables 2 and 3with average PUOFF timesof 9 sec and 3 sec, respectively. We observe that under low PUactivity, SAMER outperforms CP-AT by 0.07, 6.11, 6.20, 20.12,15.71 and 26.11 percent at the 90th-percentile for distances of1, 2, 3, 4, 5 and 6 hops respectively. SAMER achieves a higherthroughput gain over CP-ATwhen the source and destinationdistance is longer. On the other hand, under high PU activity,CP-AT outperforms SAMER by 0.18, 1.11, 43.29, 6.98, 109.66,and 74.69 percent at the 90th-percentile, respectively. Onceagain, we conclude that link quality is more important thanpath stability under low intensity of PU activity, but path sta-bility becomesmore important under high intensity.
Broken links/routes. We plot the CDFs of broken links/routes for each protocol under low and high PU activity inFigs. 8e/8f and 8h/8i, respectively. Similarly to what weobserved previously, CP-AT is the most stable protocolwith the smallest number of broken links and brokenroutes; CRP has the largest number of broken routesbecause it relies only on local observations; CP-HT hasfewer broken routes than CRP, but its throughput is lowerbecause of its much longer paths. Furthermore, althoughboth SAMER and CP-HT choose long paths, SAMER has asmaller number of broken routes for the reason weexplained in Section 4.3.
4.8 Impact of Spectrum Propagation Characteristics
In [13], the authors point out that the frequencies in thelower MHz range have better propagation characteristics
and CRN routing protocols should prefer such frequen-cies, in order to reduce the number of hops and the end-to-end delay. This is a unique feature in the design of CRP,which we have ignored so far in our study, since ourassumption is that all 10 channels have the same propaga-tion characteristics.
In this section, we examine whether CRP’s performancerelatively to the other protocols can improve if the availablechannels are distributed over a large frequency band withvarying spectrum propagation characteristics, as in [13],adopting a simulation methodology similar to that in [13].We assume that the ten channels used for data transmis-sions have five different propagation distances (i.e., everytwo consecutive channels have the same propagation dis-tance) and the propagation distances decrease from 250 to90 m with a step of 40 m. We increase the SU density anduse a new topology, shown in Fig. 9, in which 169 SUs areplaced in a grid format; the distance between two verti-cally/horizontally neighboring SUs is 85 m, so that a nodecan reach its neighboring nodes even when it uses the chan-nels with the shortest propagation distance (90 m). Similarlyto in Section 4.2, we vary the average PU OFF time from 3 to10 sec and, in each simulation round, all PUs use the sameaverage PU OFF time. For each average PU OFF time, werepeat our simulation with 20 different seeds.
Throughput comparison. Fig. 10a plots the throughputresults for each protocol. We observe that SAMER and CP-HT perform better than CP-AT and CRP under low PUactivity but worse under high PU activity. The result is simi-lar to the baseline scenario in Section 4.2 with the exceptionof CP-HT which outperforms CP-AT and CRP under lowPU activity. We conclude that giving preference to channelsof longer propagation distances did not help CRP underlow PU activity compared to the baseline scenario; itsthroughput is still the lowest among the four protocols.
TABLE 2Throughput Comparison (Avg. PU OFF Time: 9 sec)
SUN ET AL.: PERFORMANCE COMPARISON OF ROUTING PROTOCOLS FOR COGNITIVE RADIO NETWORKS 1281
Under high PU activity its relative performance withrespect to the other protocols improves compared to thebaseline scenario and its throughput is similar to CP-AT’s.
Path lengths. Fig. 10b plots each protocol’s average pathlength. CRP prefers channels with longer propagation dis-tances and chooses the shortest paths among all protocols,which is consistent with its design goal. CP-AT uses an accu-mulative path metric and as a result, in this scenario, it alsochooses short paths, even though it does not explicitly takespectrum propagation characteristics into account. CP-HThas longer paths than CRP and CP-AT, since it uses a bottle-neck metric. SAMER’s path lengths are the longest because itconsiders a link’s total throughput, as the sum over all avail-able channels, and choosing a nearest neighbor providesmore available channels. The shorter paths make CP-AT andCRP perform better than the other two protocols under highPU activity, similar to what we observed previously.
Broken links/routes. Figs. 10c and Fig. 10d plot the aver-age broken links and broken routes. We observe thatSAMER has again the largest number of broken links butthe smallest number of broken routes while CRP has thesmallest number of broken links but the largest numberof broken routes. This result suggests that CRP’s linkbreaks are more likely to result in route breaks. The rea-son is that it is more difficult for CRP to find an availablechannel to switch to in the case of a link break, since itprefers the most distant neighbors, which can only bereached over the two channels with the longest propaga-tion distance. Compared with CRP, SAMER choosesnearer neighbors and has more choices in channel switch-ing when a link breaks. CP-AT has a larger number ofbroken routes than CP-HT, which is different from whatwe observed previously. This is because CP-AT also usesan accumulative path metric which prefers shorter pathsto decrease path metric values. Although shorter pathshelp CP-AT and CRP achieve higher throughput underhigh PU activity, the larger number of broken routesresults in the poor performance of CRP and CP-AT underlow PU activity.
End-to-end delay. Fig. 10e plots the delay results of eachprotocol. From Figs. 10e and 10b, we observe that CRPachieves the lowest delay (followed closely by CP-AT) byselecting the shortest paths over channels of long propa-gation distances, which is consistent with its design goal.On the other hand, although SAMER achieves the highestthroughput under lower PU activity, its delay is alwaysthe highest in this scenario because it chooses muchlonger paths than the other protocols. CP-HT’s delaylies between CP-AT’s and SAMER’s. We conclude thatchannels with heterogeneous spectrum propagation char-acteristics introduce a tradeoff between throughput andend-to-end-delay making it difficult to simultaneouslyoptimize both metrics.
Fig. 9. New simulation topology.
Fig. 10. Different spectrum propagation characteristics.
1282 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 14, NO. 6, JUNE 2015
5 TESTBED PROTOTYPING AND EVALUATION
In this section, we present a generic software architecturefor the experimental evaluation of CRN routing protocolson a testbed based on the USRP2 platform [26]. The archi-tecture provides a complete design of PHY, MAC, andnetwork layers. Basic CRN functions such as PU activity,SU periodic sensing, and channel switching capabilitiesare also supported. Based on this architecture, we proto-type and compare CP-AT, CP-HT, and SAMER on atestbed consisting of six USRP2 nodes. We describe thesoftware architecture in Section 5.1 and the hardwareconfiguration and experimental setup of our testbed inSection 5.2. Finally, we discuss the results in Section 5.3.
5.1 Architecture
As shown in Fig. 11, the software architecture consists of aData/Decision plane, a Routing plane, and a Communicationplane.
The Data/Decision plane is responsible for channelswitching and scheduling data transmissions among nodes,using either CSMA with the help of RTS/CTS control pack-ets or TDMA. Since the nodes in our testbed are half-duplex, they cannot transmit and receive simultaneously(unlike in the simulations). In the case of CSMA, RTS isused by a node A to reserve a channel with its downstreamnode B. When node B is free to receive data packets, itswitches frequency and replies with a CTS to indicate itsavailability to node A. In the case of TDMA, each nodemaintains a schedule with the Tx/Rx slots and the corre-sponding channels. Due to the high overhead of per packetchannel reservation and switching, a node transmits abatch of packets in each slot or between two RTS/CTSexchanges. A data buffer is implemented on each node tostore packets for future transmissions.
The Data/Decision plane is also responsible for maintain-ing the sensing-transmission cycle. Each node follows aschedule according to which it checks PU activity during thesensing period and sends/receives data during the datatransmission period. Due to the limited number of USRPs,we do not use USRPs as PUs, but instead we emulate PUactivity by providing each node with an input file describingPU activity in its neighborhood over time (ON/OFF inter-vals). SUs “sense” PU activity by remaining idle during eachsensing period and looking up PU activity in their input file.
The Routing plane manages the Routing Table andimplements the route discovery and route maintenance
mechanisms which we described in Section 3. The Data/Decision Plane refers to the Routing Table before forward-ing data to another node and then uses the underlying Com-munication plane to transmit data or control packets. Whenit receives a RERR packet from the Routing Plane, whichindicates the routing path is broken because of PU activity,the Data/Decision Plane cleans the data buffer.
The Communication plane is responsible for data/controlpacket exchange among neighboring nodes. While datapackets are sent over a wireless channel using USRP2, con-trol packets (RTS, CTS, RREQ, RREP, RERR, etc.) are sentvia TCP sockets over a Gigabit Ethernet interface, whichemulates an out-of-band common control channel. Further-more, similar to [34] and [35], communication on the controlchannel and the data channel is handled by two differentthreads on the host. In emulating the common control chan-nel, we only establish TCP connections between nodeswhich are neighbors in a given topology, so that broadcastpackets are received by nodes reachable according to thattopology. An all-wireless common control channel is left asa future extension.
5.2 Hardware and Experimental Setup
We implemented CP-AT, CP-HT, and SAMER on a testbedconsisting of six nodes. Each testbed node consists of a PCrunning Ubuntu 12.04 and a USRP2. Each PC has two Giga-bit Ethernet interface cards. One of them is used to connectto USRP2 and the other one is used to enable the commoncontrol channel over a Gigabit Ethernet backbone. EachUSRP2 is equipped with a half-duplex daughterboard(XCVR2450). We use a TDMA MAC protocol in our experi-ments and allow non-interfering links to transmit in thesame time slot.
In our experiments, each node can use five channels fordata transmissions. The center frequencies are 2.512, 2.513,2.514, 2.515, and 2.516 GHz. Each channel has a bandwidthof 0.2 MHz. On each node, we used a batch size of 100 pack-ets, a packet size of 500 bytes, OFDM with BPSK, and thedefault transmit power of USRP2. The sensing-transmissioncycle consists of a 1 sec sensing period and a 3 sec transmis-sion period.
Similar to our simulations, we assume there is a PU oneach data channel in each location. In our experiments,all PUs have an average ON time of 15 sec and an averageOFF time of 10, 15, and 20 sec. For each of the three aver-age PU OFF times, we use eight different seeds to gener-ate PU activity. Due to the temporal variability of thewireless environment in a real testbed, we repeat theexperiment five times for each seed. Each experimentruns for 1,200 seconds, with the first 1,000 seconds usedfor observing PU activity and the remaining 200 secondsfor data transfer.
5.3 Testbed Results
We consider three different topologies, shown in Figs. 12a,12e, 12i. For each topology, we plot the average values forthroughput, number of broken links, and number of brokenroutes. The error bars show the standard deviations.
Topology 1. The first topology is similar to the one used in[12]. Four SUs used as relays are impacted by PUs in four
Fig. 11. Testbed software architecture.
SUN ET AL.: PERFORMANCE COMPARISON OF ROUTING PROTOCOLS FOR COGNITIVE RADIO NETWORKS 1283
different locations. Similar to [12], we only consider the for-ward paths between source and destination pairs, i.e., thefour 3-hop paths from the source to the destination.
Figs. 12b, 12c, and 12d plot the average throughput,number of broken links, and number of broken routes,respectively, for the three protocols, as a function of theaverage PU OFF time. We observe that SAMER providesthe highest throughput and the performance gap is largerunder low PU activity (average PU OFF time 20 sec). Thisresult is consistent to our simulation results. Contrary toour simulation results, SAMER’s performance is notseverely impacted by high intensity of PU activity;SAMER still outperforms CP-AT and CP-HT when theaverage PU OFF time is 10 sec. This is because, in thistopology, SAMER has the same path length as the othertwo protocols; the length of all possible paths is 3 hops.Moreover, all routing paths cross locations with similarintensity of PU activity; as a result, all three protocolshave similar numbers of broken routes, as shown inFig. 12d. Therefore, SAMER achieves higher throughput,by choosing the path with the lowest amount of SU inter-ference (highest channel diversity) and/or lowest lossratio, among the available paths of equal length and simi-lar PU activity. Although the number of link breaks ishigher for SAMER (Fig. 12c), similar to our simulationsresults, many of these breaks do not result in route breaks,as we explained in Section 4.3.
We also observe that CP-AT and CP-HT perform simi-larly in this topology, as they both choose one of the
available 3-hop paths of similar PU activity, without takinginto account SU activity or link quality. This is also similarto our simulation results, when CP-AT and CP-HT bothchoose shorter paths (Figs. 3a, 3b).
Topology 2. In the second topology the number of PUlocations increases from 4 to 6. Different from Topology 1,PUs directly impact the source and destination nodes in thistopology. We assume there are five possible paths from thesource to the destination, among which three are 3-hop andtwo are 2-hop paths.
Figs. 12f, 12g, and 12h plot the average throughput,number of broken links, and number of broken routes,respectively. We observe that CP-AT achieves the highestthroughputwith the least broken links and broken routes; CP-HT performs worse than CP-AT but better than SAMER. Thisobservation agreeswith our simulation results under high PUactivity in Section 4.2. SAMER’s poor performance comesfrom its long paths and the large number of broken routes.We found that SAMER choosesmore 3-hop paths than CP-ATand CP-HT; CP-AT chooses more 2-hop paths than CP-HTand SAMER,which is similar to our observation in Section 4.4.
Differently from our simulation results, SAMER does notperform better than the other protocols when the intensityof PU activity becomes lower. This is because our testbedscale is limited and our USRPs work in half-duplex. Eventhough it takes SU contention into account, SAMER stillneeds at least two time slots to deliver a packet from thesource to the destination, which limits its advantage overCP-AT in this scenario.
Fig. 12. Testbed evaluation. For better clarity, in Figs. 12b, 12c, and 12d, 12f, 12g, and 12h, and 12j, 12k, and 12l, the data points for SAMER andCP-HT are shifted horizontally.
1284 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 14, NO. 6, JUNE 2015
Topology 3. The third topology has six PU locations,three 3-hop paths and one 2-hop path (one less thanTopology 2). Figs. 12j, 12k, and 12l plot the averagethroughput, number of broken links, and number of bro-ken routes, respectively. One interesting observation hereis that, for all protocols, the numbers of broken links andbroken routes with average PU OFF time 10 sec are lowerthan the ones with average PU OFF time 15 sec. This isbecause there is no available route for a certain period oftime under higher PU activity since this topology has themost PU locations (6) but the fewest possible paths (4)among all three topologies. We observe that CP-AT stillachieves the highest throughput with the shortest pathlengths. However, SAMER performs better than CP-HT inthis scenario. SAMER’s path length is close to CP-HT’s inthis scenario and taking link qualities into account helpsSAMER perform better than CP-HT.
6 CONCLUSION
In this paper, we conducted the first detailed empirical per-formance study of routing protocols for CRNs using boththe ns-2 simulator and a testbed based on the USRP2 plat-form. Our main findings are: i) Taking link quality andinterference among SUs into account can greatly improvethroughput and end-to-end delay under low PU activity; incontrast, path stability and path length become the domi-nant factors that affect performance under high PU activity.ii) Considering interference among SUs in the case of multi-ple flows can result in more disjoint paths and increase totalthroughput at the cost of reduced fairness. iii) Link andpath stability are not always good performance indicators.iv) For link routing metrics that ignore link quality, limitingthe path length through the use of an additive instead of abottleneck path metric typically improves performance.This conclusion does not always hold true for link quality-based routing metrics. v) Estimating spectrum availabilitybased only on local observations cannot guarantee path sta-bility. vi) With heterogeneous spectrum characteristics, it isdifficult to simultaneously optimize both throughput andend-to-end delay.
Overall, we found that the performance of routing pro-tocols in CRNs is affected by a number of factors, in addi-tion to PU activity, and different protocols perform wellunder different scenarios. Our study motivates the designof self-adaptive protocols that choose different link/pathrouting metrics in different scenarios, in an online man-ner. We plan to investigate this direction as part of ourfuture work.
ACKNOWLEDGMENTS
This work was completed while Naveen Rawat andVikramsinh Sawant were MS students at the University atBuffalo. The contents of this work are solely the responsibil-ity of the authors and do not represent the opinions or viewsof Qualcomm Atheros or Dow Jones & Co.
REFERENCES
[1] M. Cesana, F. Cuomob, and E. Ekicic, “Routing in cognitive radionetworks: Challenges and solutions,” Elsevier Ad Hoc Netw., vol. 9,no. 3, pp. 228–248, May 2011.
[2] W.-Y. Lee and I. F. Akyildiz, “Optimal spectrum sensing frame-work for cognitive radio networks,” IEEE Trans. Wireless Com-mun., vol. 7, no. 10, pp. 3845–3857, Oct. 2008.
[3] P. Setoodeh and S. S. Haykin, “Robust transmit power control forcognitive radio,” Proc. IEEE, vol. 97, no. 5, pp. 915–939, May 2009.
[4] C. Cordeiro and K. S. Challapali, “C-MAC: A cognitive MACprotocol for multi-channel wireless networks,” in Proc. IEEE2nd Int. Symp. New Frontiers Dynamic Spectrum Access Netw.,2007, pp. 147–157.
[5] J. Jia, Q. Zhang, and X. Shen, “HC-MAC: A hardware-constrainedcognitive MAC for efficient spectrum management,” IEEE J. Sel.Areas in Commun., vol. 26, no. 1, pp. 106–117, Jan. 2008.
[6] K. C. How, M. Ma, and Y. Qin, “Routing and QoS provisioning incognitive radio networks,” Elsevier Comput. Netw., vol. 55, no. 1,pp. 330–342, Jan. 2011.
[7] P. Bahl, R. Chandra, T. Moscibroda, R. Murty, and M. Welsh,“White space networking with wi-fi like connectivity,” in Proc.ACM SIGCOMM Conf. Data Commun., 2009, pp. 27–38.
[8] H. Khalife, N. M. Malouch, and S. Fdida, “Multihop cognitiveradio networks: To route or not to route,” IEEE Netw. Mag.,vol. 23, no. 4, pp. 20–25, Jul./Aug. 2009.
[9] M. Youssef,M. Ibrahim,M.Abdelatif, L. Chen, andA. V. Vasilakos,“Routing metrics of cognitive radio networks: A survey,” IEEECommun. Surveys Tuts., vol. 16, no. 1, pp. 92–109, First Quarter 2014.
[10] I. Pefkianakis, S. H. Wong, and S. Lu, “SAMER: Spectrum awaremesh routing in cognitive radio networks,” in Proc. IEEE 3rdSymp. New Frontiers Dynamic Spectrum Access Netw., 2008, pp. 1–5.
[11] Z. Yang, G. Cheng, W. Liu, W. Yuan, and W. Cheng, “Local coor-dination based routing and spectrum assignment in multi-hopcognitive radio networks,” Mobile Netw. Appl., vol. 13, nos. 1/2,pp. 67–81, Apr. 2008.
[12] X. Huang, D. Lu, P. Li, and Y. Fang, “Coolest path: Spectrummobility aware routing metrics in cognitive ad hoc networks,” inProc. IEEE 31st Int. Conf. Distrib. Comput. Syst., 2011, pp. 182–191.
[13] K. R. Chowdhury and I. F. Akyildiz, “CRP: A routing protocol forcognitive radio ad hoc networks,” IEEE J. Sel. Areas Commun.,vol. 29, no. 4, pp. 794–804, Apr. 2011.
[14] C. Xin, B. Xie, and C.-C. Shen, “A novel layered graph model fortopology formation and routing in dynamic spectrum accessnetworks,” in Proc. IEEE 1st Int. Symp. New Frontiers Dynamic Spec-trum Access Netw., 2005, pp. 308–317.
[15] Q. Wang and H. Zheng, “Route and spectrum selection indynamic spectrum networks,” in Proc. IEEE 3rd Consumer Com-mun. Netw. Conf., 2006, pp. 625–629.
[16] A. Sampath, L. Yang, L. Cao, H. Zheng, and B. Y. Zhao, “Highthroughput spectrum-aware routing for cognitive radionetworks,” in Proc. IEEE Int. Conf. Cognitive Radio Oriented WirelessNetw. Commun., 2008.
[17] A. S. Cacciapuoti, M. Caleffi, and L. Paura, “Reactive routing formobile cognitive radio ad hoc networks,” Elsevier Ad Hoc Netw.,vol. 10, no. 5, pp. 803–815, Jul. 2012.
[18] L. Ding, T. Melodia, S. Batalama, and M. J. Medley, “ROSA: Dis-tributed joint routing and dynamic spectrum allocation in cogni-tive radio ad hoc networks,” in Proc. ACM 12th ACM Int. Conf.Modeling, Anal. Simulation Wireless Mobile Syst., 2009, pp. 13–20.
[19] K. Chowdhury and M. Felice, “Search: A routing protocol formobile cognitive radio ad-hoc networks,” Elsevier Comput. Com-mun., vol. 32, no. 18, pp. 1983–1997, Dec. 2009.
[20] S. Deng, J. Chen, H. He, and W. Tang, “Collaborative strategy forroute and spectrum selection in cognitive radio networks,” inProc. IEEE Future Generation Commun. Netw., 2007, pp. 168–172.
[21] G.-M. Zhu, I. F. Akyildiz, and G.-S. Kuo, “STOD-RP: A spectrum-tree based on-demand routing protocol for multi-hop cognitiveradio networks,” in Proc. IEEE Globecom, 2008, pp. 1–5.
[22] I. Filippini, E. Ekici, and M. Cesana, “Minimum maintenance costrouting in cognitive radio networks,” in Proc. IEEE 6th Int. Conf.Mobile Adhoc Sensor Syst., 2009, pp. 284–293.
[23] J. Broch, D. A. Maltz, D. B. Johnson, Y.-C. Hu, and J. Jetcheva, “Aperformance comparison of multi-hop wireless ad hoc networkrouting protocols,” in Proc. 4th Annu. ACM/IEEE Int. Conf. MobileComput. Netw., Oct. 1998, pp. 85–97.
[24] S. R. Das, C. E. Perkins, and E. M. Royer, “Performance compari-son of two on-demand routing protocols for ad hoc networks,” inProc. IEEE INFOCOM, Mar. 2000, pp. 3–12.
[25] R. Draves, J. Padhye, and B. Zill, “Comparison of routing metricsfor static multi-hop wireless networks,” in Proc. ACM SIGCOMM,2004, pp. 133–144.
SUN ET AL.: PERFORMANCE COMPARISON OF ROUTING PROTOCOLS FOR COGNITIVE RADIO NETWORKS 1285
[26] E. Inc. Universal software radio peripheral [Online]. Available:http://ettus.com
[27] D. S. J. D. Couto, D. Aguayo, J. C. Bicket, and R. Morris, “A high-throughput path metric for multi-hop wireless routing,” in Proc.ACM 9th Annu. Int. Conf. Mobile Comput. Netw., 2003, pp. 134–146.
[28] R. Draves, J. Padhye, and B. Zill, “Routing in multi-radio, multi-hop wireless mesh networks,” in Proc. ACM 10th Annu. Int. Conf.Mobile Comput. Netw., 2004, pp. 114–128.
[29] C. E. Perkins, and E. M. Royer, “Ad hoc on-demand distance vec-tor routing,” in Proc. 2nd IEEE Workshop Mobile Comput. Syst.Appl., Feb. 1999, pp. 90–100.
[30] M. D. Felice, K. R. Chowdhury, W. Kim, A. Kassler, and L. Bononi,“End-to-end protocols for cognitive radio ad hoc networks: Anevaluation study,” Elsevier Perform. Eval., vol. 68, no. 9, pp. 859–875, Sep. 2011.
[31] I. F. Akyildiz, W.-Y. Lee, and K. R. Chowdhury, “CRAHNs: Cog-nitive radio ad hoc networks,” Elsevier Ad Hoc Netw., vol. 7, no. 5,pp. 810–836, Jul. 2009.
[32] IEEE 802.22 working group on wireless regional area networks[Online]. Available: http://www.ieee802.org/22/
[33] R. K. Jain, D.-M. W. Chiu, and W. R. Hawe, “A quantitative mea-sure of fairness and discrimination for resource allocation inshared computer systems,” Digital Equipment Corporation,Mayanard, MA, USA, Tech. Rep. DEC-TR-301, Sep. 1984.
[34] L. Ding, P. B. Nagaraju, T. Melodia, S. N. Batalama, D. A. Pados,and J. D. Matyjas, “Software-defined joint routing and waveformselection for cognitive ad hoc networks,” in Proc. IEEE MilitaryCommun. Conf., 2010, pp. 1454–1459.
[35] P. Nagaraju, L. Ding, T. Melodia, S. Batalama, D. Pados, andJ. Matyjas, “Implementation of a distributed joint routing anddynamic spectrum allocation algorithm on usrp2 radios,” in Proc.IEEE 7th Annu. Commun. Soc. Conf. Sensor Mesh Ad Hoc Commun.Netw., 2010, pp. 1–2.
[36] K. R. Chowdhury and T. Melodia, “Platforms and testbeds forexperimental evaluation of cognitive ad hoc networks,” IEEECommun. Mag., vol. 48, no. 9, pp. 96–104, Sep. 2010.
Li Sun received the BS degree in automationfrom Tongji University, China, and the MS degreein control theory and engineering from ShanghaiJiao Tong University, China. He is currently work-ing toward the PhD degree in computer scienceand engineering at the University at Buffalo, theState University of New York. His research inter-ests lie in experimental wireless networking andsmartphone networking. He is currently a studentmember of the IEEE.
Wei Zheng received the MS degree in computerscience and engineering from the University atBuffalo, the State University of New York, in2013. He is currently working toward the PhDdegree in computer science and engineering atthe University at Buffalo, the State University ofNew York. His research interests are broadly inexperimental wireless networking and machinelearning.
Naveen Rawat received the MS degree in com-puter science and engineering from the Univer-sity at Buffalo, the State University of New York,in 2013. He is currently a software engineer atQualcomm Atheros in San Jose, CA.
Vikramsinh Sawant received the MS degree incomputer science and engineering from theUniversity at Buffalo, the State University of NewYork, in 2013. His research work was on wirelessnetworking and data intensive computing.
Dimitrios Koutsonikolas received the PhDdegree in electrical and computer engineeringfrom Purdue University in 2010. He was a post-doctoral researcher at Purdue University fromSeptember to December 2010. He is currently anassistant professor of computer science andengineering at the University at Buffalo, the StateUniversity of New York. His research interestsare broadly in experimental wireless networkingand mobile computing. He is a member of theIEEE, ACM, and USENIX.
" For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.
1286 IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 14, NO. 6, JUNE 2015
