| Date post: | 07-May-2023 |
| Category: |
Documents |
| Upload: | khangminh22 |
| View: | 0 times |
| Download: | 0 times |
University of California
Los Angeles
High-Fidelity Evaluation Framework and
Application-Centric Performance Analysis of
Vehicular Networks
A dissertation submitted in partial satisfaction
of the requirements for the degree
Doctor of Philosophy in Computer Science
by
Yi Yang
2009
c© Copyright by
Yi Yang
2009
The dissertation of Yi Yang is approved.
Jack W. Carlyle
D. Stott Parker
Yingnian Wu
Rajive Bagrodia, Committee Chair
University of California, Los Angeles
2009
ii
To my parents, without whose love this dissertation might still be unfinished.
iii
Table of Contents
1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Vehicular Networks: Issues and Challenges . . . . . . . . . . . . . 4
1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.1 Realization of Application-Centric Evaluation Paradigm . 8
1.2.2 Development of High-Fidelity Vehicular Network Evalua-
tion Platform . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2.3 Evaluation of Vehicular Network Architectures, Protocols
and Advanced Intelligent Transportation Systems . . . . . 11
1.3 Roadmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2 Application-Centric Evaluation Paradigm . . . . . . . . . . . . . 14
2.1 Motivation: Application-Level Metrics Are More Direct and Reli-
able in Predicting End User Experience . . . . . . . . . . . . . . . 16
2.2 Challenges and Benefits . . . . . . . . . . . . . . . . . . . . . . . 19
2.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.1 Performance Evaluation Techniques . . . . . . . . . . . . . 23
2.3.2 Network Studies . . . . . . . . . . . . . . . . . . . . . . . . 25
2.4 Our Realization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.4.1 Utilize Hybrid Emulation Testbed . . . . . . . . . . . . . . 26
2.4.2 Integrate Transportation Simulation . . . . . . . . . . . . 27
2.4.3 Measure Objective Application-Level Metrics . . . . . . . . 28
iv
2.5 Experimental Evaluation of Application Performance with 802.11
PHY Rate Adaptation Mechanisms . . . . . . . . . . . . . . . . . 29
2.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.5.2 802.11 PHY Rate Adaptation Mechanisms . . . . . . . . . 33
2.5.3 Experiment Setup . . . . . . . . . . . . . . . . . . . . . . . 35
2.5.4 Performance Results . . . . . . . . . . . . . . . . . . . . . 37
2.5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3 High-Fidelity Vehicular Network Evaluation Platform . . . . . 49
3.1 Phase I: Incorporate Realistic Vehicular Network Environment Set-
tings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.1.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.1.2 Model Deployment of Vehicular Network Components . . . 52
3.1.3 Model Vehicle Mobility Patterns . . . . . . . . . . . . . . . 53
3.1.4 Model Wireless Channel Effects . . . . . . . . . . . . . . . 54
3.2 Phase II: Integrate Network Simulation Into Transportation Sim-
ulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.2.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.2.2 Simulation Platform Architecture and Interfaces . . . . . . 57
4 Evaluation of Multihop Relaying for Robust Vehicular Internet
Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
v
4.3 Performance Results . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5 Evaluation of Video Streaming over Vehicular Networks . . . . 81
5.1 Experiment Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.2 Performance Results . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.2.1 Demonstrate Utility and Benefits of Application-Centric
Evaluation Paradigm . . . . . . . . . . . . . . . . . . . . . 84
5.2.2 Evaluate Application Design and Implementation in the
Target Vehicular Network . . . . . . . . . . . . . . . . . . 91
5.2.3 Investigate Cross-Layer Interaction across Protocol Stack
Including Applications . . . . . . . . . . . . . . . . . . . . 93
5.2.4 Explore Correlation Between Application-level and Network-
level Performance . . . . . . . . . . . . . . . . . . . . . . . 97
5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
6 Evaluation of VANET-based Advanced Intelligent Transporta-
tion Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.1 Dynamic Route Planning . . . . . . . . . . . . . . . . . . . . . . . 100
6.1.1 Route Computation . . . . . . . . . . . . . . . . . . . . . . 101
6.1.2 Information Aging . . . . . . . . . . . . . . . . . . . . . . 102
6.2 VANET Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6.2.1 Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6.2.2 Adaptive Broadcast . . . . . . . . . . . . . . . . . . . . . . 104
vi
6.2.3 Distributed Fair Power Adjustment . . . . . . . . . . . . . 105
6.3 Experiment Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.3.1 Scenario and Parameters . . . . . . . . . . . . . . . . . . . 105
6.3.2 Performance Measures . . . . . . . . . . . . . . . . . . . . 106
6.4 Performance Results . . . . . . . . . . . . . . . . . . . . . . . . . 108
6.4.1 Integration of Network Simulation . . . . . . . . . . . . . . 108
6.4.2 Vehicle Information Benefits Route Planning . . . . . . . . 113
6.4.3 Effectiveness of Aggregation . . . . . . . . . . . . . . . . . 115
6.4.4 Effectiveness of Adaptive Broadcast . . . . . . . . . . . . . 116
6.4.5 Effectiveness of D-FPAV . . . . . . . . . . . . . . . . . . . 121
6.4.6 Relative Performance of Protocols . . . . . . . . . . . . . . 123
6.4.7 Impact of Penetration Ratio . . . . . . . . . . . . . . . . . 124
6.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
7.2.1 Extend Distributed Simulation Platform to Execute Real
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 133
7.2.2 Examine Interaction of Periodic and Event-driven Messages 133
7.2.3 Design Adaptive Protocol to Control Traffic Data Load . . 135
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
vii
List of Figures
2.1 Example vehicular network scenario for video streaming . . . . . . 17
2.2 Comparative performance of AODV and GPSR predicted by network-
level metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3 Comparative performance of AODV and GPSR predicted using
the application-level metric PSNR . . . . . . . . . . . . . . . . . . 19
2.4 A schematic of our testbed. The black lines represent RF cables,
whereas the red arrows show signal flow. . . . . . . . . . . . . . . 36
2.5 Average throughput and packet loss rate for CBR/UDP traffic
with Onoe and SampleRate across three TGn channel models (B,
D and F). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.6 Packet inter-arrival time (jitter) for CBR/UDP traffic with Onoe
and SampleRate (TGn channel model D, 85dB path loss). . . . . 41
2.7 Per-packet rate selection trace with Onoe and SampleRate (TGn
channel model D, 85dB path loss). Packets are indexed in the
order of their reception at the receiver. . . . . . . . . . . . . . . . 41
2.8 Performance of QStream (adaptive video streaming tool) with Onoe
and SampleRate (TGn channel model D, 82dB and 90dB path loss,
respectively). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.9 Throughput for file transfer traffic with Onoe and SampleRate
(TGn channel model D). . . . . . . . . . . . . . . . . . . . . . . . 45
2.10 Mean transfer delay for web traffic with Onoe and SampleRate
(TGn channel model D). . . . . . . . . . . . . . . . . . . . . . . . 46
viii
3.1 Distribution of roadside APs in the city of Oberstrass, Zurich,
Switzerland . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.2 Vehicule density variation during a 4-hour period in the city of
Oberstrass, Zurich, Switzerland . . . . . . . . . . . . . . . . . . . 54
3.3 Simulation platform architecture for the integration of transporta-
tion simulation, network simulation and applications . . . . . . . 58
3.4 Communication among transportation simulator, network simula-
tor and applications (described in Section 3.2.2.2) . . . . . . . . . 60
4.1 Spatial distribution of APs and vehicle density variation over time
in a selected region in the city of Zurich, Switzerland. . . . . . . . 67
4.2 Spatial connectivity (fraction of vehicles connected) over time with
direct access and multihop relaying strategies at different power
and rate combinations (reflecting different communication range
values). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.3 Spatial distribution of vehicles in the selected region of Fig. 4.1(a)
after first 15 minutes, 30 minutes and 1 hour in the 4-hour period
shown in Fig. 4.1(b). . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.4 Connection duration, averaged across all vehicles over 250 second
time intervals, with direct access and multihop relaying strategies
at different power and rate combinations (reflecting different com-
munication range values). . . . . . . . . . . . . . . . . . . . . . . 73
4.5 Connection duration CDF with direct access and multihop relaying
strategies at various communication range values. . . . . . . . . . 75
4.6 Disconnection duration CDF with direct access and multihop re-
laying strategies at various communication range values. . . . . . 76
ix
4.7 Path length CDF at various communication range values for mul-
tihop relaying with hop count threshold set to infinity. . . . . . . 78
5.1 Vehicular network scenario for video streaming . . . . . . . . . . . 83
5.2 Network-level performance of GPSR and AODV in terms of through-
put, delay, jitter and loss respectively in default setting . . . . . . 85
5.3 Application-level performance of GPSR and AODV in terms of
PSNR in default setting . . . . . . . . . . . . . . . . . . . . . . . 86
5.4 PSNR of GPSR and AODV with Rayleigh fading at max velocity
30m/s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.5 PSNR of GPSR and AODV at different vehicle densities with video
rate 256Kbps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.6 PSNR of GPSR and AODV at different video rates with fixed
transmission rate 11Mbps . . . . . . . . . . . . . . . . . . . . . . 92
5.7 PSNR of GPSR and AODV with ARF at different video rates . . 94
5.8 Network-level performance of GPSR and AODV with ARF at
video rate 256Kbps . . . . . . . . . . . . . . . . . . . . . . . . . . 95
6.1 Vehicle density on the freeway over time, Vissim Only vs Vissim
+ Qualnet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.2 Traffic condition on the freeway, Vissim Only vs Vissim + Qualnet 110
6.3 Travel quality experienced by vehicles over time, Vissim Only vs
Vissim + Qualnet . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6.4 End-to-end performance of broadcast transmission, Vissim Only
vs Vissim + Qualnet . . . . . . . . . . . . . . . . . . . . . . . . . 112
x
6.5 Travel quality over time, travel time information only, vehicle in-
formation only vs combination of both . . . . . . . . . . . . . . . 115
6.6 Travel quality and availability and accuracy of traffic knowledge,
aggregation vs no aggregation . . . . . . . . . . . . . . . . . . . . 117
6.7 Travel quality experienced by vehicles over time, fixed broadcast
rate vs Adaptive Broadcast . . . . . . . . . . . . . . . . . . . . . 119
6.8 Average inter-transmission interval at each vehicle with four dif-
ferent protocol parameter sets of Adaptive Broadcast . . . . . . . 120
6.9 Travel time, fixed transmission power vs D-FPAV . . . . . . . . . 122
6.10 Knowledge percentage of traffic network, D-FPAV . . . . . . . . . 122
6.11 Distribution of computed transmission power, D-FPAV vs perfect
knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.12 Travel quality experienced by vehicles over time, comparing adap-
tion at different layers . . . . . . . . . . . . . . . . . . . . . . . . 124
6.13 Gain on vehicles reaching the destination, compared to penetration
ratio = 0% . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
xi
List of Tables
4.1 Receiver sensitivity values assumed at different 802.11b transmis-
sion rates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.2 Communication range values for different 802.11b transmission
rate and power level combinations. . . . . . . . . . . . . . . . . . 69
4.3 Average and median connection duration with direct access and
multihop relaying strategies at various communication range values. 77
4.4 Average and median disconnection duration with direct access and
multihop relaying strategies at various communication range values. 77
4.5 Percentage of gain in connection duration with multihop relay-
ing relative to direct access for increasing hop count thresholds at
various communication range values. . . . . . . . . . . . . . . . . 79
5.1 Application-level performance of GPSR and AODV in terms of
other application-layer metrics in default setting . . . . . . . . . . 87
5.2 Application-level performance of GPSR and AODV with ARF at
video rate 112Kbps . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.1 Performance measures for Dynamic Route Planning . . . . . . . . 107
6.2 Protocol Parameter Values for Adaptive Broadcast . . . . . . . . 120
7.1 Application-specific metrics for Accident Alert . . . . . . . . . . . 135
xii
Acknowledgments
Thanks to Rajive Bagrodia, my advisor and committee chair, who guided me
through the journey of my graduate study; who provided considerable insights
into the research work presented in this dissertation; who had the difficult task
of encouraging me to work independently and efficiently. To my dissertation
committee members, Jack W. Carlyle, D. Stott Parker and Yingnian Wu, for
their time, patience and interest in my research. To my parents and beautiful
sister for always believing in me and not letting me give up. To Monica Wong
and Joe Old for their love, support and enthusiasm. To Mahesh Marina, Mineo
Takai, Maneesh Varshney, Zhengrong Ji, Junlan Zhou, Zhiguo Xu and Justin
Collins for their helpful discussions and suggestions. To all my friends for their
wonderful friendship.
xiii
Vita
1977 Born, Baotou, Nei Mongol, China.
1995 - 1999 Bachelor of Science, Computer Science, Fudan University,
Shanghai, China.
1999 - 2001 Master of Philosophy, Computer Science, Hong Kong Univer-
sity of Science and Technology, China.
2001 - 2003 Master of Science, Computer Science, University of California,
Los Angeles, USA.
2003 - 2009 Doctor of Philosophy, Computer Science, University of Califor-
nia, Los Angeles, USA.
Publications
Yi Yang and Rajive Bagrodia, Evaluation of VANET-based Advanced Intel-
ligent Transportation Systems, in Proceedings of the Sixth ACM International
Workshop on Vehicular Ad Hoc Networks (VANET’09), 2009.
Yi Yang, Maneesh Varshney, Shrinivas Mohan and Rajive Bagrodia, High-
Fidelity Application-Centric Evaluation Framework for Vehicular Networks, in
Proceedings of the Forth ACM International Workshop on Vehicular Ad Hoc
Networks (VANET’07), 2007.
xiv
Yi Yang, Mahesh Marina and Rajive Bagrodia, Evaluation of Multihop Re-
laying for Robust Vehicular Internet Access, in Proceedings of the First Mobile
Networking for Vehicular Environments (MOVE’07), 2007.
Yi Yang, Mahesh Marina and Rajive Bagrodia, Experimental Evaluation of
Application Performance with 802.11 PHY Rate Adapatation Mechanisms in
Diverse Environments, in Proceedings of the IEEE Wireless Communications
and Networking Conference (WCNC’06), 2006.
Zhengrong Ji, Mahesh Marina, Maneesh Varshney, Zhiguo Xu, Yi Yang, Junlan
Zhou and Rajive Bagrodia, WHYNET: A Framework for In-Situ Evaluation of
Heterogeneous Mobile Wireless Systems, in Proceedings of the First ACM Inter-
national Workshop on Wireless Network Testbeds, Experimental Evaluation and
Characterization (WiNTECH’06), 2006.
Zhengrong Ji, Yi Yang, Junlan Zhou, Mineo Takai and Rajive Bagrodia, Ex-
ploiting Medium Access Diversity in Rate Adaptive Wireless LANs, in Proceed-
ings of the Tenth Annual International Conference on Mobile Computing and
Networking (MobiCom’04), 2004.
Hao Yang, Haiyun Luo, Yi Yang, Songwu Lu and Lixia Zhang, HOURS: Achiev-
ing DoS Resilience in an Open Service Hierarchy, in Proceedings of the Thirty-
Forth IEEE International Conference on Dependable Systems and Networks (DNS’04),
2004.
xv
Abstract of the Dissertation
High-Fidelity Evaluation Framework and
Application-Centric Performance Analysis of
Vehicular Networks
by
Yi Yang
in Computer Science
University of California, Los Angeles, 2009
Professor Rajive Bagrodia, Chair
With the continuous increase and diversification of applications and services en-
abled through information collection and communication, consequently, vehicular
networks are required to optimize performance simultaneously in the many do-
mains that these networks are expected to support. In order to improve the end
user’s satisfaction of various classes of applications, we propose an evaluation
paradigm that incorporates application-level metrics into the analysis of net-
work systems to facilitate the gathering of an understanding of vehicular network
systems from the perspective of the application, as opposed to the traditional
paradigm of evaluations centered at network-level performance. This evalua-
tion paradigm serves as a foundation for developing simulation environments
and performing analysis on vehicular networks. Under the application-evaluation
paradigm, we develop a distributed simulation platform that integrates trans-
portation simulation, wireless network simulation and real applications into a
common framework. The integrated simulation platform provides high-fidelity
communication network models and realistic transportation models, allows user
xvi
level traffic simulation and wireless network simulation and is capable of per-
forming accurate, scalable, flexible and efficient evaluation of vehicular networks.
Using the evaluation framework developed, performance analysis on existing ve-
hicular network architectures and protocols are performed to better understand
the implications of operations and optimizations at various layers of the protocol
stack on the user-perceived performance for a variety of applications. Four case
studies are designed and conducted, which, in light of application-level and trans-
portation system level metrics, respectively examine the effectiveness of vehicular
networks in supporting file transfer, web browsing, video streaming, Internet ac-
cess and Intelligent Transportation Systems.
xvii
CHAPTER 1
Overview
Recent surveys show that traffic congestion costs are staggering: total amount
delay of 3.7 billion hours, wasted fuel of 2.3 billion gallons and annual costs of
$63 billion to the US economy [59]. Such menacing price being paid, it becomes
essential to recognize the critical link between technology and management of
transportation infrastructure – apply advanced technologies for information ac-
quisition, analysis and application to the management of transportation systems.
Intelligent Transportation Systems (ITS) that exploit in-vehicle information tech-
nology (e.g. mobile computing and wireless communication) in surface trans-
portation systems are an example of emerging technologies to reduce the impact
of traffic congestions. Vehicle-to-vehicle and vehicle-to-roadside communication
systems, known as vehicular networks that interconnect onboard sensing and
communication devices, are an important component of advanced transportation
systems such as ITS and useful for a wide variety of applications that include in-
cident detection, crash reporting, traveler information dissemination and media
content sharing. In-vehicle sensors and communication devices offer the poten-
tial for detailed and accurate data collection (e.g. second-to-second position,
speed, acceleration and deceleration) and information transfer and allow cover-
age to extend beyond areas where roadside equipment has been placed. The key
components of vehicular networks are the mobile vehicles on the road and fixed
access points (APs) along the roadside. Such wireless networks are interesting
1
and challenging to study due to the unique characteristics of these networks which
distinguish them from existing network architectures such as MANET, wireless
mesh, WLAN, etc.
As emerging vehicular networks extend existing roadside infrastructures (e.g.
sensors, access points and centralized servers), it is imperative to understand
their impact on end-user applications that will utilize these networks and more
importantly to isolate fundamental performance limitations of vehicular networks
in order to expand their applicability to support appropriate higher level appli-
cation services. Test bed environments are an important component that may
be used to evaluate new techniques, system designs and architectures. However
experimentation in operational transportation systems is costly, can be danger-
ous, does not scale well and often does not provide sufficient means of control for
comprehensive experimentation. Simulated systems that include detailed models
of transportation and communication are capable of overcoming these limita-
tions. As both traffic conditions and network communication patterns are highly
variable and unpredictable, accurate and efficient simulation tools are vital not
only to assess the benefits of vehicular network based transportation systems in
a planning mode but also to generate scenarios, optimize control and predict net-
work behavior at the operational level for transportation professionals to develop
effective traffic management systems and to compare transportation alternatives.
However, models adopted in the current suite of network simulators fail to cap-
ture traffic conditions in a very faithful fashion. Take mobility for example, the
commonly-used models are directly borrowed from studies of wireless ad hoc net-
works, which inaccurately represent real-life vehicle movement patterns. Thus it
is crucial that simulation environments that provide high-fidelity transportation
models and allow user level traffic simulations and wireless network simulations
to be integrated in a common framework be in place in order to perform accurate,
2
scalable, flexible and efficient evaluation of vehicular networks.
Various services and applications are enabled through information collection
and communication in vehicular networks. The primary class of applications that
are supported by vehicular networks are safety applications such as accident alert
and congestion warning. Non-safety and commercial applications are becoming
equally important for improved efficiency, productivity and convenience while
commuting. Examples of these applications include advanced traveler informa-
tion systems that are part of ITS, legacy Internet applications (e.g. e-mail, web
browsing, file downloading and audio/video streaming), location-based services
(e.g. gas price, restaurant specials) and on-road commercial applications (e.g.
media sharing, online gaming). With the continuous increase and diversification
of applications and services, consequentially, vehicular networks are required to
optimize performance simultaneously in the many domains that these networks
are expected to support, as the quality of service requirements of various appli-
cations and services can be significantly different. For example, accident alert
requires very low latency and authenticated communication within a local area;
while a movie downloading service requires high bandwidth and low jitter com-
munication from the roadside to vehicles. To improve the user (i.e. drivers and
passengers on the road) satisfaction of various classes of applications, a primary
requirement is to gather an understanding of vehicular network systems from
the perspective of the application, as opposed to the traditional paradigm of
evaluations centered at network-layer metrics. Achieving such objective requires,
foremost, an evaluation tool that incorporates application-level metrics into the
analysis of network systems thus enabling the measurement of user satisfaction.
This evaluation tool serves as a foundation for performing analysis on existing
vehicular network architectures and protocols to better understand the implica-
tions of operations and optimizations at various layers of the protocol stack on
3
the application-level perceived performance.
Using the developed evaluation platform, in light of application-level met-
rics, the current generation of vehicular network architectures and protocols are
revisited. The goals of our performance evaluation include:
• Examine the effectiveness of vehicular networks in supporting a variety of
applications and services.
• Understand operation of protocols in large scale and high mobility vehicular
network settings.
• Quantitatively measure application-level performance differences of alter-
native choices of architectures and protocols.
• Provide insights for design of adaptive cross-layer optimizations.
The following sections of this chapter first discuss the motivations and chal-
lenges of the research problems stated and the justifications for devoting research
efforts in this field. Then, the contributions that have been achieved in this dis-
sertation are presented. Last, the organization for the rest of the dissertation is
described in Section 1.3.
1.1 Vehicular Networks: Issues and Challenges
There are several possible network architectures to organize and connect in-
vehicle systems and roadside units. Three alternatives include a pure wireless
vehicle-to-vehicle ad-hoc network (i.e. VANET), a wired backbone with wireless
last-hops (i.e. infrastructure-based), or a hybrid architecture using vehicle-to-
vehicle communication that does not rely on a fixed infrastructure but can exploit
it for improved performance and functionality when it is available.
4
Being a particular type of wireless networks, vehicular networks share common
characteristics with other existing wireless network architectures. Some of these
characteristics include:
• Short range wireless communication technologies are typically employed
as the wireless communication technology in vehicular networks, including
IEEE 802.11 and Direct Short Range Communication (DSRC). This deter-
mines that vehicles and APs have limited direct communication range, up
to a few hundred meters.
• Bandwidths of vehicular network links are in the order of tens of Mbps (e.g.
11Mbps with 802.11b, 54Mbps with 802.11a/g).
• Channel conditions encountered by a vehicular network vary depending on
the environment in which the vehicular network operates, including rural,
suburban and urban areas, highways and local streets.
• Vehicular networks are mobile.
• Except for the infrastructure formed by roadside APs, the wireless networks
constituted by moving vehicles are ad hoc.
• Vehicles have low storage capacity.
• Data compression/aggregation can be required to accommodate for the lim-
ited bandwidth of the wireless medium.
Besides the above properties that vehicular networks have in common with
other types of wireless networks, they also exhibit unique features that distinguish
them from the rest. Some of these features pose challenges for both performance
evaluation and protocol design in vehicular networks. Listed below are a subset
of them.
5
• Given the large number of vehicles on the road, vehicular networks are
inherently large scale. As a result, the evaluation technique used to study
vehicular networks is required to be capable of modeling large scale networks
in an efficient manner. Further, network protocols and system components
developed for vehicular networks are demanded to be scalable to the size
of the networks.
• Vehicular networks are highly dynamic because of relatively fast movements
of vehicles. Link topologies of vehicular networks change rapidly, causing
fast variations in the network connectivity. Frequent network disconnec-
tions can happen, especially in the case of low vehicle density, where the
gap between two vehicles might be several miles. As a consequence, ve-
hicular network protocol/optimization design should target at developing
adaptive solutions that are responsive to such highly dynamic environments.
• Under vehicular network environments, wireless channel quality is affected
by vehicle velocity, which can be relatively high (e.g. 80miles/hour on high-
way). In city scenarios, the existence of large number of obstructions cre-
ates complicated channel conditions, causing communication to experience
multi-path fast fading effect and resulting in bursty lossy links. Conse-
quently, to conduct accurate analysis of network performance, high-fidelity
channel models should be used. Similarly, protocol design should take into
account the wireless channel environment in which the target vehicular net-
work operates.
Although some of the unique properties of vehicular networks make their
study and design difficult, other features of these networks can be utilized to
assist optimization and design.
6
• Geographical information of vehicles can be obtained through GPS devices,
including location, direction of movement and speed. Such geographical
information assists the operation and possibly optimization of vehicular
network protocols, an example of which is geographical routing protocols.
• Vehicular networks provide the feasibility of partially predicting vehicular
positions since vehicles normally run among pre-build roads that remain
unchanged over the years. Movements of vehicles are constrained by road
maps, regulations and behaviors of other vehicles around them.
• Energy is not a big issue since a vehicle itself can be used as a source of
electric power.
1.2 Contributions
This research effort attempts to make several contributions to evaluation tools
and performance studies of vehicular networks, including
• Proposing an application-centric evaluation framework for vehicular net-
works by utilizing a hybrid emulation testbed, incorporating transportation
system level measures and measuring objective application-level metrics.
• Integrating high-fidelity models of vehicular network environment into the
application-centric evaluation framework.
• Developing a distributed simulation platform that integrates transportation
simulation, wireless network simulation and applications into a common
framework.
• Performing application-centric studies of vehicular network architectures
and deployment of advanced ITS systems such as Dynamic Route Planning
7
and commercial applications such as video streaming on vehicular networks;
comparing the performance of alternative protocols proposed to support
these applications under realistic operating conditions of vehicular networks
in light of application-level and system-level metrics.
1.2.1 Realization of Application-Centric Evaluation Paradigm
The application-centric evaluation paradigm incorporates, into the analysis of tar-
get networks, measures that are capable of interpreting the impact of a network
architecture or system components on end-user experience. We first describe the
results from a case study that highlights the importance of this application-centric
outlook of evaluation. This case study illustrates the effectiveness of application-
centric metrics over network-centric metrics by considering video streaming appli-
cation over a vehicular network scenario operating with different routing protocols
(i.e. AODV and GPSR).
We then discuss the multiple reasons why the application-centric paradigm of
evaluation is desirable. To summarize, first, the user-level satisfaction is related
to application-centric metrics in a very direct fashion since the latter directly
encapsulates the end performance measures that we are interested in. Network-
centric metrics, however, suffer from the problem that there are multiple such
metrics that can be considered and the relations between them are not clearly
understood. Second, even if the relation between these network metrics can be
understood, the correlation of these metrics with application centric metrics are
not always trivial. Finally, network-centric evaluation is agnostic to application
layers and the obtained results have to be re-interpretated for different classes of
applications.
However, realization of such application-centric evaluation is challenging. First,
8
high-fidelity models of applications are required in the evaluation of networks.
Second, transportation system level metrics such as travel time and delay time
are not provided by the current suite of network simulators. Third, the subjec-
tivity associated with application-level metrics makes them difficult not only to
measure but also in reproducing the results.
We have addressed the above challenges by a three-fold approach. The diffi-
culty in modeling the applications is alleviated by using operational applications
in the context of a hybrid emulation testbed [72]. This approach dispenses the
need to model applications and allows measuring statistics from actual imple-
mentations of the applications. The lack of transportation system level measures
is overcome by integrating transportation simulation into existing network sim-
ulators. The third problem of the subjectivity of application-layer metrics is
addressed by enabling the substitution of these subjective metrics with objective
ones that are still very close in semantics to the former.
To further verify the effectiveness of application-centric performance evalu-
ation, we examine the impact of physical layer rate adaptation mechanisms on
the performance of real applications over 802.11 wireless links in diverse channel
environments. Our evaluations are based on a testbed with real wireless devices
equipped with commodity 802.11 hardware and a hardware channel emulator. We
consider two different and well-known 802.11 rate adaptation mechanisms (Onoe
and SampleRate) and study their performance under several realistic workloads,
including multimedia streaming and web browsing. We observe that the appli-
cation performance with different rate adaptation mechanisms is dependent on
the specific tradeoffs these mechanisms make at the link layer in an application-
oblivious manner between improving throughput and limiting frame loss. More
importantly, their relative performance for a given workload is quite sensitive to
9
the channel quality and environment. These observations highlight the impor-
tance of choosing the rate selection strategy adaptively in an application and
channel aware manner.
1.2.2 Development of High-Fidelity Vehicular Network Evaluation
Platform
In order to build a high-fidelity evaluation framework for vehicular networks,
at the first step of our development, we extend the application-centric evalua-
tion paradigm to achieve accurate, scalable, flexible and repeatable performance
studies through the utilization of a hybrid emulation testbed and incorporation of
high-fidelity protocol and environment models. The proposed evaluation frame-
work not only addresses the unique challenges of vehicular networks but also en-
ables new types of network analyses via the capability of conducting application-
centric evaluation.
To further enhance the capabilities of the evaluation platform, a distributed
simulation platform that integrates transportation simulation, wireless network
simulation and applications into a common framework is proposed and imple-
mented. The evaluation platform provides both realistic transportation system
models and high-fidelity wireless network communication models for accurate
and efficient evaluation of vehicular networks. Further, the evaluation platform
facilitates dynamic cyclic interaction between the two simulation domains, al-
lowing runtime control of vehicles’ behavior in the transportation simulation as
they react in real time to information exchange in the simulated communication
network. Using the evaluation platform, realization of emerging vehicular net-
work applications and services such as Intelligent Transportation Systems can
be achieved with high degree of realism, providing a user level simulation envi-
10
ronment to evaluate the feasibility and performance limitations of VANETs in
supporting these applications and services.
1.2.3 Evaluation of Vehicular Network Architectures, Protocols and
Advanced Intelligent Transportation Systems
The last part of this dissertation focuses on performance evaluation of vehicular
network architectures, protocols and advanced ITS in light of application-centric
and system-level evaluation metrics within a high-fidelity model of vehicular net-
work operating environment. Analysis of the network refers to the study of the
influence on a given application class by the design of the protocol stack and dif-
ferent vehicular scenario configurations. The objective is to discover the delivered
application level performance by a specific network architecture and protocol.
The first case study is concerned with the design of network architecture
that better suits the application level needs, i.e. how the choice of the network
architecture affects the application performance. Given an application to be sup-
ported, AP deployments in the vehicular network and vehicule mobility patterns,
the analysis tries to answer the question of how should the communication be-
tween the vehicles and APs be organized to optimize the user-level performance.
We study connectivity benefits of using a multihop relaying strategy for improved
Internet access in a WiFi-based vehicular environment relative to the common
strategy that allows only direct communication between vehicles and access points
(APs). We use real AP location data and realistic and detailed vehicular mobility
traces for our study. Our results show that multihop relaying strategy leads to
substantial gains in connectivity relative to direct access as much as 400%, and
that multihop relaying combined with increased communication range provides
even greater gains (up to 467%). Further, relay paths with few hops are sufficient
11
to realize most of the gain with multihop relaying.
The second case study is devoted to the analysis of the protocol stack. We
consider protocols at the MAC layer (rate adaptation), network layer (routing
protocols) and transport layer, and study the influence of the various alternatives
at these layers of the protocol stack on the application performance. We also con-
sider the influence of channel and mobility parameters together with these proto-
cols on the application-level performance. The purpose of this study is to under-
stand the relative benefits and drawbacks of different designs of protocols when
considering application-layer performance. Compared to traditional network-
centric evaluation, case studies show scenarios where network-level statistics do
not clearly discriminate between the two routing protocols while significant per-
formance differences were observed using the application-level metrics. i.e. order
of tens dB on PSNR improvement and 38.3% reduction on mean square root of
error achieved by AODV over GPSR.
The third case study is conducted using the proposed distributed simulation
platform to evaluate the performance of Dynamic Route Planning when deployed
in VANETs using metrics collected at the transportation system level such as
travel and delay time. The effectiveness of three representative VANET dynamic
adaptation protocols in enhancing the application performance in scenarios with
high vehicle density are compared in the case studies. The experiment results
show that Dynamic Route Planning can be effectively supported by a VANET
system with up to 118% increase on the number of vehicles reaching the destina-
tion, 36.2% reduction on travel time and 56.1% reduction on delay time.
12
1.3 Roadmap
The rest of the dissertation is organized as follows. In Chapter 2, we introduce
the application-centric evaluation paradigm and quantitatively justify the bene-
fits of application-level evaluation by various case studies. In Chapter 3, we de-
scribe the evaluation platform of vehicular networks which provides high-fidelity
transportation models and integrates the network simulation into transportation
simulation. The methodology proposed in Chapter 2 will be used in Chapter 4
and 5 for the analysis of vehicular network architectures and protocol stack. The
distributed simulation platform proposed in Chapter 3 will be used in Chapter 6
to evaluate the performance of Dynamic Route Planning deployed on VANET.
13
CHAPTER 2
Application-Centric Evaluation Paradigm
One of the key proposals presented in this dissertation is to provide a framework
and mechanisms that can incorporate, into the analysis of vehicular networks,
measures that are capable of interpreting the impact of various transportation
system architectures and alternative system components and protocols on the
experience of a driver or a passenger on the road. In other words, this chapter
of the dissertation is an attempt to shift the paradigm of performance evalua-
tion of vehicular networks from the conventional network-centric perspective to
an application-centric perspective. In general, network-centric evaluation is tar-
geted at studying the dynamics of communication, i.e. packet transmission, and
examines how such dynamics evolve over time or how they are influenced by sce-
nario or system parameters. The metrics used in network-centric evaluation, as
commonly reported in current research literature, include packet delivery ratio,
aggregate throughput, end-to-end delay, jitter etc. Application-centric evalua-
tion, in contrast, assimilates these dynamics into measures of application-layer or
system-level performance from the perspective of an end user who determines the
level of satisfaction of the perceived network operation. It should be noted that
while the network-centric evaluation is agnostic of upper layer applications and
the obtained results are claimed to be applicable for network operation in general,
the application-centric evaluation is contingent on the specific application(s) that
is supported by the network. For instance, an application-level metric for web
14
browsing performed by a passenger in a car may be the time interval between the
passenger requesting a URL by clicking a web link and him beginning to see the
web page in the browser. For a streaming video application that plays streamed
clips from the Internet on the display within a vehicle, an application-level met-
ric could be the duration when the user-observed video is distorted or there are
‘glitches’ in the video, or the number of times the video freezes. Similarly, in
Dynamic Route Planning, the travel quality experienced by a driver can be man-
ifested in metrics such as travel time and delay time that are collected at the
transportation system level.
In the following sections, an example study case is first presented to quanti-
tatively demonstrate the importance and advantages of using application/system
level metrics in the evaluation of vehicular networks. Next, the associated chal-
lenges and benefits of application-centric evaluation are described, followed by the
discussion of related work on existing evaluation techniques and network stud-
ies from the perspective of application-centric evaluation. Then, our proposed
approaches for realizing this mode of evaluation are presented. Finally, to fur-
ther verify that application-centric evaluation achieves more accurate analysis in
terms of user-perceived application quality, the comparative performance of two
representative rate adaptation protocols is studied from the perspective of both
the network-level and the application-level metrics for a variety of widely-used
applications.
15
2.1 Motivation: Application-Level Metrics Are More Di-
rect and Reliable in Predicting End User Experience
To demonstrate that compared to network-level metrics, application-level metrics
more truthfully reveal the network performance experienced by an end user, a case
study is presented in this section, where the network-level and the application-
level performance of a video streaming application in a vehicular network are
contrasted. The scenario in the case study consists of vehicles moving on a
freeway. The vehicles are equipped with wireless devices and able to communicate
with each other (refer to Figure 2.1). Along the freeway there are base stations
that serve as gateways to the Internet. The vehicles can connect with these base
stations, possibly over multi-hop routes, to access the Internet. The application
used is a media player that displays on a client inside a vehicle a streaming video
from a server on the Internet. AODV and Greedy Perimeter Stateless Routing
(GPSR) are used as the alternative underlying routing protocol in the vehicular
network. The network-level metrics considered include throughput, delay, jitter
and loss. Peak Signal to Noise Ratio (PSNR) is used as the application-level
metric of the video streaming application.
Figure 2.2 shows the throughput, delay, jitter and loss of using AODV and
GPSR respectively as the underlying routing protocol in the vehicular network. A
glance at these results does not immediately reveal the comparative performance
of the video streaming application that can be expected by the end users. Further,
by these results, it is difficult to determine the use of which of the two routing
protocols would result in better video quality at a client.
Figure 2.3 plots the observed PSNR for each frame of video clip received at
the client. The maximum value of PSNR, which corresponds to no distortion of
16
Internet
RoadsideBase-station
Client
Vehicle Router
Vehicular Network
Figure 2.1: Example vehicular network scenario for video streaming
video, is 100dB. The shaded areas in the graphs can be viewed as time instances
when the video is corrupted and the magnitude indicates the extent of corruption.
With a look at this graph one can easily discern how the video application is
performing over time. Clearly, AODV outperforms GPSR in delivering higher
quality of video to a client.
In conclusion, performance results in this case study quantitatively illustrate
that application-level metrics are more directly related to end user experience and
therefore facilitate the process of understanding the impact of the operation of a
particular vehicular network system on the application-level performance. For a
video streaming application, the network-level metrics bear little correlation with
the application-level performance measures. In general, the possible correlation of
network-level metrics with application-level metrics closely relates to the specific
implementation details of the application. This provides another argument for
using application-level metrics in the network performance evaluation for the
purpose of obtaining reliable evaluation results.
17
0 10 20 30 40 500
200
400
600
800
Time(sec)
Thro
ughp
ut (K
bps)
GPSRAODV
(a) Throughput
0 10 20 30 40 500
2
4
6
8
10
12
Time (sec)De
lay
(sec
)
GPSRAODV
(b) Delay
0 10 20 30 40 50!2
!1
0
1
2
Time (sec)
Jitte
r
GPSRAODV
(c) Jitter
0 10 20 30 40 500
200
400
600
800
Time (sec)
Loss
GPSRAODV
(d) Loss
Figure 2.2: Comparative performance of AODV and GPSR predicted by net-
work-level metrics
18
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(a) GPSR
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(b) AODV
Figure 2.3: Comparative performance of AODV and GPSR predicted using the
application-level metric PSNR
2.2 Challenges and Benefits
Application-centric metrics, based on the level of human involvement in the pro-
cess of measuring them, can be categorized into two broad classes: objective and
subjective/perceptual metrics. Objective application-centric metrics refer to the
set of measures at the application-level, the estimation of which does not require
an end user’s subjective assessment but can be directly computed from statistics
collected at the application layer. For example, the aforementioned application-
level metric of web browsing can be easily measured by time-stamping the in-
stances at which a user clicks a web link to request a URL and when he begins to
see the corresponding web page in the browser. For more complex applications,
however, the metrics can be subjective to human interpretations and therefore,
demand direct human involvement. An example of such applications is streaming
video, in which, application-level performance, i.e. the video quality perceived
by an end user, is subjected to that user’s very own interpretation. How well a
user is satisfied with the experienced video quality depends on personal factors
19
including the individual’s perceptual capabilities, past experiences with video
streaming etc.
The realization of application-centric evaluation presents difficulties due to
multiple reasons. First, a high fidelity model of an application is required to
enable the measurement of application-centirc metrics, both objective and sub-
jective. Some applications such as web browsers, FTP sessions etc can be modeled
with relative ease and are already part of the current suite of network simula-
tors. For a large set of applications like video streaming, IP telephony and video
conferencing, however, abstraction of these applications into models is very com-
plicated.
Second, for a large percentage of vehicular network applications, including
Intelligent Transportation Systems (ITS) such as Cooperative Collision Avoid-
ance Systems and Dynamic Route Planning, the measurement of effectiveness of
these applications being supported by VANET manifests in performance metrics
at the transportation system level, the statistics of which are not provided by
conventional network simulators. For example, the effectiveness of deploying Dy-
namic Route Planning on VANET networks in terms of driver experienced travel
quality can be measured by metrics like travel time, delay time, vehicle density
and speed, and traffic volume. Metrics like these at the transportation system
level more closely reflect the interest of transportation system planners, providers
and consumers.
Last, subjectivity associated with some application-level metrics makes them
not only hard to measure but also difficult in reproducing results. The most
common, and perhaps the most accurate, method employed to measure subjec-
tive metrics is to arrange volunteers to observe a live performance of the target
application, operating under the dynamics of the network, and collect data from
20
each observer regarding the subjective rating of the performance. A statistical
aggregation of the collected data is considered as a representation of the user
perception of evaluation. The applicability and usefulness of this approach is,
however, restricted by the following challenges:
• Arranging a session of evaluation with a group of observers is prohibitively
costly in the time and resources required.
• Such an evaluation requires real applications that the observers can interact
with and also an operational network that influences the performance of
these applications, both of which can be difficult or costly to realize. For
example, the cost of implementing a vehicular network testbed is high and
it is nearly impossible to recreate scenarios.
• The feedback loop between observing the performance and adjusting the
protocol parameters or modifying the design is too slow.
• The results may be difficult to reproduce since they are, in the end, sub-
jective.
These challenges and difficulties of application-centric evaluation, associated
with the relative ease in obtaining measurements of network-level metrics, have
been the reason why application-centric evaluation has failed to be considered
as a criteria for evaluation studies. However, we argue that if these challenges
can be successfully addressed, this paradigm of evaluation will offer benefits in
network analysis. We present here some of these benefits:
• Simplicity. While network-level metrics are easy to obtain, there exist a
multitude of these metrics, each reflecting the performance of the vehicular
network protocol or system under study on a single aspect of lower layer
21
network dynamics. It is hard to correlate a set of network-level metrics to
the final application-level performance observed by end users. Therefore,
it becomes complicated to understand the user-perceived performance to
be expected at the application or system level given comparative perfor-
mance of different protocols or systems on various network-level metrics.
Application-level metrics, on the other hand, are direct reflection of end
user satisfaction, the use of which in turn extricates the comprehension of
performance analysis results.
• Correctness. In the case study presented in the previous section, it is ob-
served that while the network statistics do not clearly discriminate between
the two routing protocols under investigation, there are in fact significant
differences that can be observed by the application level metrics. Results
like these illustrate that the network analysis conducted using only network-
level metrics may not correctly reveal the performance of applications that
can be expected by end users. This observation applies to a variety of appli-
cations with protocols operating at different layers, which is demonstrated
by results from case studies in Section 2.5 and Chapter 5. These experi-
ences in network evaluation demonstrate that application-centric evalua-
tion achieves more reliable performance analysis by using application-level
metrics and hence provides developers more accurate understanding of the
operation of vehicular network protocols and systems so that correct mod-
ification or optimization can be considered.
• Facilitation of cross-layer design and optimization. Operation and detailed
configuration of network protocols and systems are driven by the achieved
level of user satisfaction of the applications which these protocols and sys-
tems attempt to support. In analysis of these protocols and systems, quanti-
22
tatively measuring user-perceived performance at application level provides
insights into (1) the impact of operation of network protocols and systems
on the performance of applications (2) the interaction between the respec-
tive adaptations or optimizations operating at application and lower layers.
Such insights if obtained enlighten future cross-layer design and optimiza-
tion.
2.3 Related Work
2.3.1 Performance Evaluation Techniques
The performance evaluation technique to be employed to perform application-
centric evaluation plays a critical role in determining the feasibility and easiness
of measuring application-level metrics. Currently, three categories of evaluation
techniques exist: simulation, physical testbed and emulation. In the following
subsections, capabilities and limitations of these platforms are discussed from
the perspective of being utilized to realize application-centric evaluation.
2.3.1.1 Simulation
The most commonly used technique to model and analyze wireless networks is
discrete event simulatiors (e.g. QualNet [49], NS-2 [38], Glomosim [13], Opnet [42]
and PDNS [51]). The simulation paradigm models packet-level communication
of wireless networks and is, therefore, an excellent candidate for the purpose of
collecting network-level metrics. While simulation tools offer a flexible and scal-
able approach to create reasonably detailed physical and link models, and as well
possess repeatability and controllability, simulation models cannot be used for
application-centric performance evaluation as they do not provide the capability
23
of executing operational softwares (real implementations of applications). For
this reason, simulation, especially traditional simulation tools, is unlikely to be
the choice of evaluation technique for application-centric evaluation.
2.3.1.2 Physical Testbed
Physical testbeds can certainly address the preceding shortcoming of simulation
by having operational applications. Although they can capture cross-layer inter-
actions between real applications down to real physical links, physical testbeds
suffer from other limitations. First, the resource investments needed to deploy
a large scale physical testbed make it prohibitive to use physical testbed as the
tool to evaluate wireless networks that are inherently large scale, such as vehic-
ular networks. In addition, it is difficult to provide repeatable experiments for
a given input configuration, particularly with diverse operating conditions like
vehicles on a highway. Physical testbeds, by their very nature, require the net-
work protocol/system under study to be physically realized, which makes them
unsuitable for performance analysis of futuristic design of protocol/system that
may not yet be available for deployment.
2.3.1.3 Emulation
Emulation refers to the evaluation techniques where the protocol stack for an
emulated host in the network contains real implementations from one layer up to
the application layer, while lower layers such as PHY and MAC are simulated.
Clearly, real applications are able to run in emulation, which is essential to realize
application-centric evaluation.
24
2.3.2 Network Studies
In the existing literature, measurement of application-level performance is con-
strained to studies that aim at designing better applications or application-layer
adaptations and optimizations. In [66] and [29], rate-distortion performances of
various video coding standards are compared in light of video-specific metrics
including PSNR and Just Noticeable Difference (JND) [33]. Little work has been
done to incorporate application-level metrics into network performance evalua-
tion and conduct network analysis from the application-centric perspective. In
few network studies, application-level metrics are introduced; however, the target
applications are simple and these metrics do not involve or reflect any end user
subjectivity, making the measuring of them easy to realize. For example, in [11],
application-level latency metrics such as packet inter-reception time (IRT) and
cumulative number of packet reception are defined to measure the performance
of a collision warning application.
2.4 Our Realization
In our proposal, the challenges associated with application-centric evaluation are
addressed by a three-fold approach. The difficulty in accurately modeling the
applications is averted by using operational applications through a hybrid emula-
tion testbed TWINE [72]. The resulted evaluation paradigm allows the execution
of real applications like media player in a simulated network environment. The
lack of metrics at the transportation system level is overcome by the integration
of an established transportation simulator with a network simulator. The issues
in conducting perceptual evaluation are addressed by enabling the substitution
of subjective application-level metrics with objective ones that relatively easier
25
to measure but still close in semantics to the former.
2.4.1 Utilize Hybrid Emulation Testbed
TWINE [72] is a hybrid emulation testbed developed at UCLA Mobile Systems
Lab (MSL), which combines simulation, emulation and physical networks in an
integrated testbed for evaluation of wireless protocols and systems. Emulation
refers to evaluation techniques where the protocol stack for an emulated host in
the network contains real implementations of the higher layers while lower layers
such as PHY and MAC are simulated. The integration of emulation and simu-
lation in TWINE enables perceptual evaluation of real applications over a wide
range of wireless network scenarios. TWINE is also highly scalable to permit eval-
uation of large-scale target networks like vehicular networks in lab environment.
It contains high fidelity models of lower layers and physical environment (e.g.
channel), which provide realistic details particularly at wireless physical layer as
well as the flexibility to support diverse network conditions. Experiments can
be repeated with the same set of controlled parameters so as to support fair
comparison among different network protocols and systems.
Provided with the advantages of TWINE, our first approach is to move the
vehicular network evaluation paradigm from simulation or testbeds to emulation-
based evaluation. This emulation-based framework is integrated into our eval-
uation paradigm and tailored for vehicular network performance studies. More
specifically, in order to assess application-level metrics, a client or end-host is
emulated, i.e. being operational to run real applications that interface with the
operating system. This client or end-host communicates with other hosts in the
network using an emulated network interface. In addition to being able to run
operational applications, realizing a client or end-host in emulation facilitates the
26
capture of real time interactions between applications and lower layers. Simula-
tion is used to model the rest of the network, the operation of which does not need
to be modeled at the same level of fidelity as the emulated hosts, thus enabling
evaluation of large scale vehicular networks. When prototype or implementa-
tion of a specific network protocol or system is available, a subnet of operational
nodes can be created through emulation, where the actual code can be evaluated.
High-fidelity channel and host mobility models are integrated as well to create
realistic environment settings for the vehicular network under investigation.
2.4.2 Integrate Transportation Simulation
The existing set of established transportation simulators provide the functionality
of collecting statistics on a variety of transportation system level metrics includ-
ing travel time, delay time, vehicle density and speed, queue length in delay,
traffic volume etc, which are necessary to capture the application-level perfor-
mance of applications such as Dynamic Route Planning. In order to incorporate
these metrics into our evaluation paradigm, a distributed simulation platform
is developed (for detailed discussion, see Section 3.2), which uses a standard
TCP connection to dynamically link a microscopic transportation simulator (e.g
VISSIM [62]) and a packet-level network simulator (e.g. QualNet [49]). The real-
ized simulation platform facilitates, in additional to conventional network-centric
metrics, the measurement of performance metrics at the application and trans-
portation system level, which more closely reflect the interest of transportation
system planners, providers and consumers.
27
2.4.3 Measure Objective Application-Level Metrics
As discussed earlier in the chapter, perceptual estimation of application-level
metrics is difficult to arrange and time costly. In order to expedite the evaluation
process and improve efficiency, objective application-level metrics that closely
reflect user-perceived performance are employed in our paradigm. For example,
we propose the use of PSNR as the application-level objective metric for streaming
video, which is highly correlated with the subjective discernment of humans, thus
serves as a good candidate of measures of the user-perceived video quality. PSNR
is defined as the ratio of the maximum possible power of a video signal and the
power of corrupting noise that affects the fidelity of the signal’s representation
in the received video. For streaming video, the “the original signal” is the video
stored at the server and the “noise corrupted signal” is the video shown at the
client. How much of the final version of the video differs from the original is
captured by PSNR and is usually expressed in dB. Low dB values indicate that
there is a lot of corruption in the video frames. Our focus lies in developing
methods so that the measurement of objective metrics such as PSNR can be
effectively integrated into network performance studies. This is demonstrated in
the following paragraph via an example in which PSNR for streaming video is
measured. Other work-in-progress applications include IP telephony and video
conferencing.
In our measurement of PSNR, VLC [63] is used as a representative of video
streaming applications. Two emulated hosts serve as streaming video server and
client, running VLC in server and client mode respectively. Streaming video
originates from the operational server, travels through a simulated multi-hop
vehicular network, and is finally delivered at the client. The received video is
played at the operational client and can be visually monitored to make judgement
28
of the user-percieved video quality. Video frames can possibly be dropped by the
simulated network as well as discarded from being displayed by VLC client due to
late arrival. This final displayed video clip is captured and stored at the client for
post-processing. The original video clip at the VLC server and the final displayed
one at the client are divided respectively into a sequence of video frames using
software VirtualDub [61] so that PSNR of each user-oberserved video frame can
be computed. Each pair of the original and final displayed video frames with the
same frame index are then passed as input parameters to Wavelet (a class library
for wavelet transforms on images [65]) which computes the distance of the video
image pair in PSNR. In the process, statistics on other application-level metrics
such as corrupted frames, number of corrupted intervals, average duration of
corrupted intervals, corrupted sec per minute of video and dropped frames can
also be easily collected.
2.5 Experimental Evaluation of Application Performance
with 802.11 PHY Rate Adaptation Mechanisms
To verify that application-centric evaluation achieves more accurate network anal-
ysis in terms of the user-perceived application quality, the comparative perfor-
mance of two representative IEEE 802.11 rate adaptation protocols are studied
in this set of experiments from the perspective of both network-level metrics, e.g.
throughput, loss, delay and jitter, and application-level metrics for three types
of widely-used applications, including spatial and temporal quality of adaptive
video streaming, application-level throughput of file transfer and mean transfer
delay of web traffic. A key observation from the study is that the relative perfor-
mance of the two rate adaptation mechanisms are significantly different depend-
ing on the performance measure. This manifests that to more correctly analyze
29
user-perceived performance from network protocols/systems, application-centric
evaluation should be embraced.
2.5.1 Introduction
Wireless LAN (WLAN) technology based on IEEE 802.11 standard [20] is being
commonly used in offices and hotspots for indoor wireless Internet access. The
success of 802.11 technology has led to newer usage scenarios, including com-
munity mesh networks [6] and multimedia distribution/data networking in the
home. Given the widespread use of 802.11-based networks, it is critically impor-
tant to understand the performance characteristics of wide range of applications
(with different QoS requirements) on such networks when operating under diverse
channel environments.
In this set of experiments, our focus is on 802.11 physical layer (PHY) data
rate adaptation mechanisms that are usually implemented in the 802.11 PHY
layer. Such a mechanism adjusts the data rate in response to time-varying channel
conditions: the basic idea is to exploit good channel conditions by using higher
rates for improved efficiency (throughput), and improve transmission reliability
when channel gets worse by lowering the rate. The 802.11 PHY provides several
widely different data rates (differing in modulation and coding) for use by higher
layers — 802.11b rates range from 1 to 11Mbps, whereas 802.11a/g extend this
range to 54Mbps. In such settings, clearly the PHY rate adaptation (via dynamic
rate selection) plays a key role in determining performance observed at higher
layers. This together with the fact that 802.11 MAC/PHY specifications leave
the rate adaptation mechanism to vendor discretion led to many proposals to
optimize throughput with some constraint on frame error rate (FER).
Several recent studies have been directed toward experimental evaluation of
30
802.11 PHY rate adaptation mechanisms [5,15,22,28,70]. Notwithstanding their
contribution to the understanding of real-world performance of various rate con-
trol mechanisms, these studies have one main limitation: they are largely based
on throughput measurements with backlogged UDP traffic. At first glance, this
may seem reasonable given that the primary motivation behind rate adaptation is
throughput improvement by taking advantage of good channel conditions. How-
ever, this metric alone is not sufficient to predict application layer performance in
general as it may also depend on additional metrics such as frame loss rate; these
metrics in turn are affected by the interactions among rate adaptation, MAC
ARQ mechanism, frame length etc. Moreover, some of these studies [15, 28] are
specific to a certain channel environment (specifically, an office environment),
whereas the rest of them are limited in their experiment control (in terms of
configurability and reproducibility) or realism.
In this set of experiments, our goal is experimental characterization of the in-
teraction between applications and PHY rate adaptation mechanisms over 802.11
wireless links in different environments. Towards this end, we take a unique ap-
proach through the use of a hardware channel emulator [47]. Such a channel
emulator provides us with a highly realistic testbed due to its detailed signal-
level emulation of the wireless channel. In addition, it offers high degree of
control in terms of experimenting with a wide range of channel conditions in a
repeatable manner. For our evaluations, we consider three different channel en-
vironments representing home, office and suburban usage scenarios respectively;
these environments are realized using a subset of channel models being consid-
ered by the 802.11 Task Group n (TGn) [19]. Further, the real-time nature of
the emulator permits application-level performance evaluation when real wire-
less devices (running real applications) such as laptops with commodity 802.11
hardware communicate via the emulator.
31
We focus on FER-based rate adaptation mechanisms (e.g., [5, 23, 28, 35]) in
this set of experiments as they are relatively more practical than the alternative
set of SNR-based mechanisms (e.g., [18,53]) and naturally robust across different
channel environments [28]. In particular, our evaluations consider Onoe [35]
and SampleRate [5, 6] as two representative rate adaptation mechanisms. We
choose these two specific mechanisms as they were not only shown to be the most
effective among FER-based mechanisms [5] but also are sufficiently different in
their design.
As per applications, we use a broad set of realistic workloads consisting of
CBR/UDP traffic, adaptive video streaming [27], large file transfers and web
(HTTP/TCP) traffic. Together these workloads cover dominant types of traffic
characteristics of 802.11 networks [17], albeit indirectly in some cases. For in-
stance, we do not directly consider peer-to-peer (p2p) traffic which amounts to
nearly 20% of overall workload in [17]; instead it is captured via file transfer and
web traffic in our workloads, which is reasonable given that most p2p data traffic
uses TCP either directly or via HTTP [56].
Our results indicate that the PHY rate selection strategy must be adaptively
chosen based on both channel quality and application characteristics for best over-
all application-level performance. In the process of optimizing raw throughput
at the link layer in an application-independent manner, different rate adaptation
mechanisms (with different levels of aggressiveness) provide markedly different
tradeoffs with the number of packet losses seen by higher layers. Consequently,
their throughput and loss rate behaviors at the link layer can be quite differ-
ent. The net effect of this tradeoff on the application performance depends on
the application-specific characteristics as might be expected. We quantify the im-
pact of this tradeoff for Onoe and SampleRate on performance of several common
32
applications under realistic channel environments.
More interestingly, the observed performance of a given application with dif-
ferent rate adaptation mechanisms is quite sensitive to the channel quality and the
environment with neither Onoe nor SampleRate exhibiting superior performance
throughout. Specifically, significant performance reversals are seen between the
two mechanisms across different regions of the parameter space. For instance,
SampleRate performance of TCP bulk file transfer with TGn channel model D is
5x better than Onoe for low path loss values, whereas it becomes 10x worse for
very high path loss; we observe similar patterns for other workloads as well. The
above observations suggest that throughput optimization at the link layer in an
application-oblivious manner is ineffective when designing PHY rate adaptation
mechanisms. Rather these mechanisms in conjunction with other MAC opera-
tions (including frame size selection, scheduling and ARQ) must be attuned to the
application requirements when adapting to varying channel conditions, further
emphasizing the need for cross-layer optimization.
The rest of the section is organized as follows. In Section 2.5.2, we briefly
describe the two rate adaptation mechanisms used in our study, namely Onoe
and SampleRate. Section 2.5.3 describes our experimental setup. Section 2.5.4
presents our results and forms the core of the case study; this section is further
divided into several subsections according to the type of workload. We conclude
in Section 2.5.5.
2.5.2 802.11 PHY Rate Adaptation Mechanisms
Existing 802.11 PHY rate adaptation mechanisms can be classified into two broad
categories depending on how they estimate the channel quality. In FER-based
mechanisms [5,23,28], the channel quality is implicitly estimated from FER (or re-
33
lated) measurements obtained by intermittent probing at higher rates (much like
the way TCP probes by increasing the sending rate to estimate available band-
width). On the other hand, SNR-based mechanisms [18, 53], explicitly estimate
the channel quality using the physical layer information (e.g., SNR) typically at
the receiver. Here we limit our attention to FER-based mechanisms as they are
relatively more practical and naturally robust across different channel environ-
ments [28], and leave comprehensive evaluation also involving SNR-based schemes
and hybrid schemes [15] for future work. Among the FER-based mechanisms, we
consider Onoe [35] and SampleRate [5, 6] as two representative rate adaptation
mechanisms because they were not only shown to be the most effective in their
category [5] but also are sufficiently different in their design. Below we briefly
review these two mechanisms to serve as background for the rest of the section.
2.5.2.1 Onoe
Onoe [35] is the default rate adaptation mechanism in all wireless cards based on
Atheros chipsets. It is a credit-based strategy in that it maintains credits for the
currently used rate on a per-destination basis to aid in the decision to increase
the data rate. Initially, the rate is set to 24Mbps for 802.11a/g, and 11Mbps for
802.11b, with zero credits for that rate. Subsequently, Onoe decides on the rate
and updates credits periodically every observation interval (default one second)
based on success of frame transmissions and retransmission count during the
previous observation interval. It steps down to a next lowest rate if either none of
the transmissions were successful in the previous interval, or more than ten frames
were transmitted with average retries exceeding one. Credit count is decremented
if more than 10% of the frames retried during the previous observation interval,
and incremented otherwise. If the credit count reaches a threshold (10) then Onoe
34
shifts to a next higher rate. Clearly, Onoe is conservative in moving to higher
rates — it takes at least ten observation intervals for a rate increase, whereas
rate decrease can happen in just one interval.
2.5.2.2 SampleRate
This mechanism lays special emphasis on lossy links (typical in outdoor environ-
ment). The design of SampleRate [5, 6] is based on the following insight: for
lossy links, using higher data rates with higher loss rates can result in higher raw
link layer throughput; in other cases, highest data rate with low loss rate gives
highest throughput. SampleRate works as follows. It selects the data rate on a
per-frame basis. Initially, it uses the highest possible rate (11Mbps in 802.11b
and 54Mbps in 802.11a/g). Afterwards, the eligible rate with smallest estimated
average frame transmission time (including loss recovery) is selected. A data rate
is eligible if successful transmission time without retry using that rate is smaller
than average transmission time of current rate and use of that rate did not result
in four consecutive transmission failures. To estimate the frame transmission
times at eligible rates other than the current rate, every tenth frame is sent using
a data rate randomly chosen from the set of eligible rates excluding the current
rate.
2.5.3 Experiment Setup
Figure 2.4 illustrates our testbed setup. We use the PROPSim C8 wideband mul-
tichannel simulator [47] for fine-grained wireless channel emulation. By default,
this channel emulator supports only one-way channels. Since 802.11 MAC re-
quires bidirectional communication (for exchanging DATA and ACK frames), we
enable such two-way communication using a pair of one-way channels provided
35
Laptop 1 (AP)
PROPSim C8 Channel Simulator
Laptop 2 (Wireless Host)
Figure 2.4: A schematic of our testbed. The black lines represent RF cables,
whereas the red arrows show signal flow.
by the emulator and a combination of two-way splitters, isolators and different
types of connecters. Two laptops (Dell Latitude D600 model) form the end hosts
for the wireless link in our testbed. Both laptops are equipped with a commodity
802.11 wireless PC cards having an external antenna port (Proxim ORiNOCO
Gold 802.11b/g) in order to connect to the channel emulator using RF cables.
To minimize leakage, we shield the cards by wrapping them with copper foil.
Proxim cards we used are based on Atheros chipsets for which open-source
linux drivers are available. In particular, we use the widely used Multi-band
Atheros Driver for WiFi (MADWiFi) [35], which already includes implementa-
tions for Onoe and SampleRate rate adaptation mechanisms. Laptops in our
testbed run Fedora 2.6.10 kernel. We configured the laptops in 802.11 infras-
tructure mode such that one of them acts an access point (AP) and the other as
the wireless host (see Figure 2.4). We use 802.11b in all our experiments with
RTS/CTS disabled. We use the default MAC retry limit (4) with same rate
36
across all retransmissions.
We use a subset of TGn channel models [19] in our experiments to represent
diverse environments typical of 802.11 deployments. Specifically, we consider the
following three models: (i) Model B with 15ns rms delay spread — residential
home or small office environment; (ii) Model D with 50ns rms delay spread —
a typical office environment, non-line-of-sight (NLOS) conditions; (iii) Model F
with 150ns rms delay spread — a large open space (indoor and outdoor) envi-
ronment, NLOS conditions. For each of the above models, we further vary the
path loss value in our evaluations (using a parameter in the channel emulator) to
create a wide range of channel conditions. Throughout we use a fixed transmit
power (10dBm) at the sender.
2.5.4 Performance Results
2.5.4.1 CBR/UDP Traffic
We begin our study of the interaction between applications and rate adaptation
mechanisms with UDP performance of Onoe and SampleRate in different envi-
ronments. The UDP workload is useful in several respects. As already mentioned,
most previously reported results are based on UDP traffic. So using an identical
type of workload allows us to relate results from our testbed with those in the
literature. Using UDP traffic also eases the analysis of the relationship between
throughput and other network-centric metrics (e.g., loss rate) of importance to
applications because it does not entail application or transport layer adaptation.
Finally, it is indicative of the non-adaptive multimedia workloads.
We generate UDP traffic from the AP to the wireless host in our testbed (see
Figure. 2.4) using the Multi-Generator (MGEN) [39], a well-known open-source
37
software from the PROTEAN research group at NRL. Besides traffic generation,
MGEN also includes a tool called DREC for logging and analyzing statistics at
the receiver. We examined four metrics: throughput, loss rate, packet latency
and packet inter-arrival time (jitter). We observed latency performance to be
similar to that of throughput in all our experiments. So we do not show latency
results in the interest of space. We experimented with a wide range of traffic loads
(between 1Mbps and 7Mbps) by varying the packet generation rate in MGEN
while keeping the packet size fixed (1000 bytes). Since the performance behaviors
we report are seen across all loads, here we present results only for one traffic
load (4Mbps) for brevity.
Figure 2.5 shows the relative performance of Onoe and SampleRate in terms
of average throughput and packet loss rate for three different TGn channel models
(B, D, F). For each channel model, we exercise different rate adaptation mech-
anisms over a wide range of channel conditions by varying the path loss value.
Unless otherwise mentioned, each data point in the figures corresponds to an aver-
age value taken over a 180 second period. From Figure. 2.5, it can be clearly seen
that the channel model (environment) heavily influences the region and extent of
performance differentials between the two mechanisms.
SampleRate consistently outperforms Onoe in terms of throughput (Figure. 2.5
(a), (c), (e)) with progressively greater improvements as we go from channel model
B to D to F; relative performance of SampleRate over Onoe also gets better as we
transition from low to intermediate path loss values. The throughput differences
between Onoe and SampleRate can be attributed to their design differences. Re-
call from Section 2.5.2.1 that Onoe uses a conservative credit-based strategy to
increase the data rate — sufficient number of credits (10) must be accumulated
before a higher rate is tried, which can happen only when less than 10% frames
38
0
1000
2000
3000
4000
5000
70 75 80 85 90 95
Througput (Kbps)
Path loss (dB)
OnoeSampleRate
(a) TGn channel model B
0
0.2
0.4
0.6
0.8
1
70 75 80 85 90 95
Loss fraction
Path loss (dB)
OnoeSampleRate
(b) TGn channel model B
0
1000
2000
3000
4000
5000
70 75 80 85 90 95
Througput (Kbps)
Path loss (dB)
OnoeSampleRate
(c) TGn channel model D
0
0.2
0.4
0.6
0.8
1
70 75 80 85 90 95
Loss fraction
Path loss (dB)
OnoeSampleRate
(d) TGn channel model D
0
1000
2000
3000
4000
5000
70 75 80 85 90 95
Througput (Kbps)
Path loss (dB)
OnoeSampleRate
(e) TGn channel model F
0
0.2
0.4
0.6
0.8
1
70 75 80 85 90 95
Loss fraction
Path loss (dB)
OnoeSampleRate
(f) TGn channel model F
Figure 2.5: Average throughput and packet loss rate for CBR/UDP traffic with
Onoe and SampleRate across three TGn channel models (B, D and F).
39
are retried in ten successive observation intervals (each of one second duration).
So Onoe is much more likely to stay at the current rate even if it experiences a
small percentage of frame losses. SampleRate, on the other hand, has a much
weaker constraint on frame losses to use higher rates — a higher rate is used
provided that rate has a smaller average frame (re-)transmission time than the
current rate, which is gleaned from continuous probing at promising alternate
rates.
Relative performance of Onoe and SampleRate with respect to loss rate is
exactly opposite from that of throughput (Figure. 2.5 (b), (d), (f)) — Onoe always
has a lower loss rate in all channel models with greater reductions seen as path
loss value increases. This behavior can be explained using similar explanation as
above. In particular, Onoe’s conservative use of lower rates relatively improves
its ability to provide higher reliability of frame transmissions, hence fewer frame
losses that go unrecovered by the MAC ARQ mechanism.
Onoe exhibits much better jitter performance compared to SampleRate (Fig-
ure. 2.6). In particular, the packet inter-arrival time with Onoe varies over a
small range [5, 20]ms, whereas it goes up to 60ms with SampleRate. As seen
from Figure. 2.7, Onoe remains steady at a rate that provides a high probability
of successful frame transmission. SampleRate, on the other hand, is aggressive in
trying higher rates and potentially risks frame losses in the process. This in turn
triggers the 802.11 MAC ARQ mechanism (backoff and retransmission). The ex-
ponential latencies due to the backoff strategy explain the frequent large spikes
seen in the packet inter-arrival time with SampleRate.
40
0
10
20
30
40
50
60
0 20 40 60 80 100 120 140 160 180
Interarrival (millisec)
Time (sec)
avg over time window 0.5s
(a) Onoe
0
10
20
30
40
50
60
0 20 40 60 80 100 120 140 160 180
Interarrival (millisec)
Time (sec)
avg over time window 0.5s
(b) SampleRate
Figure 2.6: Packet inter-arrival time (jitter) for CBR/UDP traffic with Onoe and
SampleRate (TGn channel model D, 85dB path loss).
0
2
4
6
8
10
12
14
0 5000 10000 15000 20000
Rate (Mbps)
Packet Index
tx rate
(a) Onoe
0
2
4
6
8
10
12
14
0 10000 20000 30000 40000 50000
Rate (Mbps)
Packet Index
tx rate
(b) SampleRate
Figure 2.7: Per-packet rate selection trace with Onoe and SampleRate (TGn
channel model D, 85dB path loss). Packets are indexed in the order of their
reception at the receiver.
41
2.5.4.2 Adaptive Video Streaming
Having looked at traffic that reflects non-adaptive streaming of multimedia in the
previous subsection, we now move to adaptive streaming media applications that
adapt media quality in response to changing network conditions. In best-effort
service networks like the Internet, adaptive media streaming when running on top
of TCP-friendly congestion control protocols allows for efficient sharing of net-
work resources (e.g., bandwidth) while gracefully adapting media quality. Recent
characterization studies of streaming video traffic [60] find that currently the bulk
of the streaming traffic uses TCP as opposed to UDP. Based on this observation,
our study uses a TCP-based, publicly available adaptive video streaming tool
called QStream [27, 48]. Briefly, QStream uses a scalable video compression for-
mat called SPEG (an easier to implement variant of MPEG-4 FGS) and follows a
priority drop strategy for application-level adaptation. In particular, it prioritizes
video data units based on their relative importance and drops low priority units
right at the sender when available network bandwidth falls (as reported by the
underlying congestion control protocol, TCP in this case)
QStream supports two measures for assessing video quality corresponding to
spatial and temporal dimensions. Spatial quality represents the spatial resolution
of a video frame; SPEG codec (used by QStream) provides four spatial quality
levels. Temporal quality is the frame rate in terms of the number of frames
displayed per second (fps).
We studied QStream performance with Onoe and SampleRate across different
channel models and a wide range of path loss values for each model. However
due to space constraints, we present only a small subset of that data to illustrate
our main point that neither rate adaptation mechanism outperforms the other for
all channel models and path loss values. Figure. 2.8 shows the QStream perfor-
42
mance observed during a three minute period (when a stored video is streamed
between the two laptops in our testbed) for two path loss values (82dB and 90dB,
respectively) using TGn channel model D.
With a low path loss value (82dB), SampleRate exhibits superior average and
peak performance over Onoe across both measures (Figure. 2.8(a),(c)). We no-
tice a reversal in relative performance when the channel quality gets worse with
increase in path loss value to 90dB (Figure. 2.8(b),(d)); this is evident from the re-
peated pauses seen with SampleRate across both measures. We also observe that
SampleRate performance shows wide variability in cases where it performs well,
whereas Onoe has relatively steady albeit poor performance (Figure. 2.8(a),(c)).
The above differences in performance behaviors with SampleRate and Onoe can
be traced back to their differences in throughput, loss rate and jitter perfor-
mance seen earlier with UDP traffic, and the effect of loss on the TCP sending
rate. Further, the poor performance with both mechanisms (although at differ-
ent operating regions) highlights their ineffectiveness in providing good overall
performance; this is rooted in their inability to tune the adaptation strategy in
response to the channel quality and application characteristics.
The above performance issue can be addressed with a combination of tech-
niques at different layers (e.g., unequal error protection at all layers; use of FEC at
application/link layers; loss differentiation at transport layer; use of incremental
redundancy, and judicious choice of frame sizes and retransmission limits at the
link layer) along with inter-layer awareness. An alternative and potentially more
promising approach would be to more tightly integrate all layers and jointly opti-
mize them toward best application performance. Thorough comparison of these
different approaches is left for future work.
43
0
1
2
3
4
5
0 20 40 60 80 100 120 140 160 180
Spatial quality
Time (sec)
OnoeSampleRate
(a) Spatial Quality, 82dB
0
1
2
3
4
5
0 20 40 60 80 100 120 140 160 180
Spatial quality
Time (sec)
OnoeSampleRate
(b) Spatial Quality, 90dB
0
5
10
15
20
25
0 20 40 60 80 100 120 140 160 180
Temporal Quality
Time (sec)
OnoeSampleRate
(c) Temporal Quality, 82dB
0
5
10
15
20
25
0 20 40 60 80 100 120 140 160 180
Temporal Quality
Time (sec)
OnoeSampleRate
(d) Temporal Quality, 90dB
0
1
2
3
4
5
0 20 40 60 80 100 120 140 160 180
Transmission rate (Mbps)
Time (sec)
OnoeSampleRate
(e) Transmission Rate, 82dB
0
1
2
3
4
5
0 20 40 60 80 100 120 140 160 180
Transmission rate (Mbps)
Time (sec)
OnoeSampleRate
(f) Transmission Rate, 90dB
Figure 2.8: Performance of QStream (adaptive video streaming tool) with Onoe
and SampleRate (TGn channel model D, 82dB and 90dB path loss, respectively).
44
0.01
0.1
1
10
70 75 80 85 90 95Througput (Mbps)
Path loss (dB)
OnoeSampleRate
Figure 2.9: Throughput for file transfer traffic with Onoe and SampleRate (TGn
channel model D).
2.5.4.3 File Transfer Application
In this subsection, we focus on the more traditional application of file transfer; it
also shares the characteristics of file downloads in emerging P2P applications. A
key characteristic of this application is its zero tolerance to packet losses, which
necessitates the use of a reliable transport protocol like TCP. The application
layer throughput is the primary measure of performance for file transfer. We use
the well-known Netperf tool [41] for generating a large TCP workload to reflect
file transfer traffic.
Figure. 2.9 shows the file transfer performance (throughput) with Onoe and
SampleRate for TGn channel model D for a wide range of path loss values. We
can observe that SampleRate performs much better than Onoe when the channel
quality is good (i.e., low path loss values) while it is the opposite at higher
path loss values. For instance, the throughput with SampleRate is a factor of five
greater than Onoe at 70dB path loss, whereas it is one-tenth that of Onoe for 95dB
45
0
5
10
15
20
25
70 75 80 85 90 95Mean Transfer Delay (sec)
Path loss (dB)
OnoeSampleRate
Figure 2.10: Mean transfer delay for web traffic with Onoe and SampleRate (TGn
channel model D).
path loss. This is interesting given that the main motivation behind SampleRate
design is to improve throughput under lossy conditions [5]. As noted before, the
higher loss rate with SampleRate at high path loss values (as clearly seen with
UDP traffic) has a dominating impact on the sending rate of TCP, hence the
poor application throughput. The results in this section reiterate the need for
application-aware PHY rate adaptation.
2.5.4.4 Web Traffic
Finally we consider web (HTTP) traffic which dominates the traffic both in the
Internet as well as 802.11 networks [17]. Even though web traffic is also based on
TCP like file transfer, it has two unique characteristics that make it quite differ-
ent. First, it is characterized by short-lived flows. Second, it is more interactive
in nature. The second characteristic directly relates to the key performance mea-
sure of interest (from a user viewpoint) when evaluating web traffic, i.e., the time
it takes to complete the transfer of an object from the point a request is made
46
(transfer delay). We generate the web traffic using the SURGE tool [3], which
is based on data collected from empirical measurements. Specifically, we setup
a web server on the AP in our testbed and have the other wireless host act as a
user making web requests (of varying file sizes). We use the default parameter
settings of SURGE with HTTP/1.1. As seen from Figure. 2.10, the web traffic
performance (mean transfer delay) exhibits similar relative performance behav-
iors between Onoe and SampleRate as before with similar underlying causes.
Comparing results in Figure. 2.9 and Figure. 2.10 shows that it is important
to consider application-level characteristics for realistic evaluations. Even though
both file transfer and web traffic are based on TCP, application performance with
Onoe and SampleRate in both cases is quite different. For file transfer traffic,
SampleRate is 5x better than Onoe at 70dB path loss, whereas it is 10x worse
than Onoe at 95dB path loss; for web traffic, on the other hand, performance
differentials are around factor of three at both those path loss values.
2.5.5 Summary
In the case studies presented in the previous sections, we have experimentally
studied the interplay between the performance of real applications and the de-
sign of PHY rate adaptation mechanisms over 802.11 wireless links under different
channel environments. A key aspect of our study is the use of highly realistic
testbed based on a real-time hardware channel emulator. Using this testbed, we
have evaluated a wide range of realistic application workloads (including adap-
tive video streaming and web traffic), some of them having multiple metrics,
over two representative 802.11 rate adaptation mechanisms, namely Onoe and
SampleRate.
Across all our workloads, we have found that neither rate adaptation mech-
47
anism consistently outperformed the other. Further, their performance can be
significantly different with the relative performance dependent on the channel
characteristics and the performance measure. This is in sharp contrast to prior
evaluations based on raw throughput performance using backlogged UDP traf-
fic. Fundamentally, these differences are rooted in the way different mechanisms
tradeoff loss rate when optimizing link layer throughput without consideration
for application characteristics. Thus, our results specifically highlight the impor-
tance of application-awareness in determining the adaptation strategy for rate
selection in response to time-varying channel conditions, and more generally the
need for cross-layer optimization.
Our future work will focus on extending this evaluation study to include
other types of rate adaptation mechanisms, and developing more effective link
adaptation mechanisms based on cross-layer awareness.
48
CHAPTER 3
High-Fidelity Vehicular Network Evaluation
Platform
This chapter presents our effort in building a high-fidelity evaluation platform for
vehicular networks in order to achieve accurate and efficient evaluation of various
classes of applications and services to be deployed in these networking systems.
At the first phase of our design and development, we extend the application-
centric evaluation paradigm proposed in Chapter 2 to tailor the requirements of
vehicular networks. Specifically, using the hybrid emulation testbed TWINE [72],
which combines simulation, emulation and physical networks in an integrated
testbed, evaluation of large scale vehicular networks can be performed in lab
environment. Further, we incorporate into the framework high fidelity models
of lower protocol layers and physical environments that are specific to vehicu-
lar networks (e.g. channel, mobility, deployment data), which provide realistic
details as well as the flexibility to support diverse network conditions. The ad-
vantages of the proposed evaluation framework are manifested not only in the
scalability and accuracy of the obtained results, but also in the types of analyses
that are enabled. The latter part, referred to as application-centric evaluation in
the previous chapter, provides an understanding of the performance of vehicular
networks at the application level, which is essential to improve the experience
of drivers and passengers on the road, i.e. their satisfaction of perceived ser-
49
vices provided by the underlying vehicular networks. The resulted evaluation
platform therefore addresses the unique challenges of vehicular networks and the
issues with the existing evaluation techniques – physical testbeds and simulation,
and provides accurate, scalable, flexible and repeatable performance studies of
realistic large-scale vehicular networks. A case study, which uses this evaluation
platform to investigate the performance of a video streaming application running
in a vehicular network scenario, is presented in Chapter 5 to further demonstrate
the utility and benefits of the evaluation platform.
At the second phase of our development, we propose a distributed simulation
platform that allows runtime control of vehicle behavior by events generated by
an application as a result of information exchange in the communication net-
work. The proposed simulation architecture uses a standard TCP connection to
dynamically link a microscopic transportation simulator i.e. VISSIM [62] and
a packet-level network simulator i.e. QualNet [49]. Such TCP-based simulator
coupling provides cross operating system communication necessary for these two
simulation packages running in a distributed fashion on separate computers con-
nected via LAN. To make TCP connection feasible, a program called VISSIM
Control is developed, which interacts with the Component Object Model (COM)
interface provided by VISSIM to control the transportation simulation. Mean-
while, VISSIM Control establishes TCP socket and communicates with QualNet
on behalf of VISSIM. The proposed simulation platform facilitates, in additional
to conventional network-centric metrics, the measurement of performance metrics
at the application and transportation system level, which more closely reflect the
interest of transportation system planners, providers and consumers. Using the
simulation platform, case studies are designed and conducted to investigate how
VANETs can significantly expand the deployment of Intelligent Transportation
Systems, which will be discussed in Chapter 6.
50
3.1 Phase I: Incorporate Realistic Vehicular Network En-
vironment Settings
Past research has shown that the models of network environment have a sig-
nificant impact on the results of wireless network analysis [40, 71]. To achieve
accurate performance evaluation of vehicular networks, it therefore becomes im-
perative that environment settings reflect realistic scenarios. In order to create
realistic environment settings in the specific context of vehicular networks for
the protocols and systems under study, high fidelity models of wireless channel
and vehicle movements, as well as real deployment data of roadside APs, are
integrated into our application-centric evaluation paradigm to enable accurate
evaluation. Below, we discuss in detail each of these aspects.
3.1.1 Related Work
Based on the evaluation technique used, the past performance studies in the
context of vehicular networks can be divided into two broad categories: measure-
ments and simulation. Using small-scale physical testbeds, a large set of mea-
surement studies [7,12,43,69] examine the wireless communication characteristics
of packet transmission among vehicles and between vehicles and roadside APs.
Although physical testbeds allow real implementations to be tested, the resource
investments needed to deploy a large scale physical testbed make it prohibitive to
use physical testbeds as the tool to evaluate vehicular networks in a scalable man-
ner. Further, it is difficult to provide repeatable experiments for a given input
configuration via physical testbeds, particularly with diverse operating conditions
like vehicles on a highway. In contrast, past simulation studies [2, 21, 37, 40, 68]
are conducted to investigate the performance of various vehicular network pro-
51
tocols, mainly running at network and MAC layer. While simulation offers a
flexible and scalable approach, models adopted in simulation evaluation may fail
to capture network deployment and vehicle movement patterns in a very faith-
ful fashion. Take mobility for example, the commonly-used models are directly
borrowed from studies of wireless ad hoc networks, which inaccurately represent
real-life vehicle movement patterns. Further, simulation tools do not provide the
flexibility of executing operational softwares (e.g. real application implementa-
tions). These observations indicate that in order to perform accurate, scalable,
flexible and repeatable evaluation of vehicular networks, an evaluation frame-
work that utilizes the relative benefits of both physical testbeds and simulation
is required to be in place.
3.1.2 Model Deployment of Vehicular Network Components
To configure vehicular network deployment in a realistic manner, real road maps
of selected regions are used in our settings. Locations (e.g. GPS coordinates) of
the currently deployed roadside APs are obtained through AP locator websites
like www.jiwire.com and www.wifimaps.com and used as the AP positions in the
experiments. In the case study presented in Chapter 4, location information of
APs in Zurich, Switzerland is integrated into the analysis framework and used
to set up the network evaluation. A total number of 321 WiFi hot spots were
found, including their addresses, location types, access providers and wireless
technologies employed. Figure 3.1 shows the distribution of the roadside APs in
the city of Oberstrass, Zurich, Switzerland, which is used as the selected region in
our study. Deployment of APs in other areas and countries can also be obtained
in a similar way.
52
6.8 6.81 6.82 6.83 6.84 6.85x 105
2.42
2.43
2.44
2.45
2.46
2.47
2.48
2.49 x 105
X Coordinate
Y Co
ordi
nate
Figure 3.1: Distribution of roadside APs in the city of Oberstrass, Zurich, Switzer-
land
3.1.3 Model Vehicle Mobility Patterns
It is imperative, for an accurate performance evaluation of vehicular networks,
that the mobility patterns should reflect the realistic scenarios. First, we have suc-
cessfully incorporated vehicle mobility traces [40] into our evaluation framework.
The mobility traces are obtained from a detailed vehicular movement simulation
over real road maps using MMTS [40]. It contains a 24-hour movement pattern
(coordinates, moving directions and speeds) of a total number of 259, 978 vehi-
cles in Switzerland (area of 41, 559km2). The traces are further parsed to obtain
movement data for selected regions and scenarios (e.g., city, highway) and with
desired vehicle density and speed. Figure 3.2 shows the vehicle density variation
over a 4-hour period in the city of Oberstrass, Zurich, Switzerland.
To further improve the accuracy of vehicle mobility and provide online ve-
hicular movement generation, we have integrated a microscopic vehicular traffic
simulator VISSIM [62] into the network simulator Qualnet [49]. Using real city
maps, combined with VISSIM, close to reality vehicular movement patterns are
53
0 1 2 3 40
500
1000
1500
2000
2500
3000
3500
4000
Time (hour)
Num
ber o
f Act
ive
Vehi
cles
Figure 3.2: Vehicule density variation during a 4-hour period in the city of Ober-
strass, Zurich, Switzerland
able to be produced. Vehicles in VISSIM are mapped to mobile nodes in Qual-
Net. In real time, the generated movement patterns are transferred from VISSIM
to QualNet, which accordingly simulates wireless communication among vehicles.
The details of the integration are described in Section 3.2.
3.1.4 Model Wireless Channel Effects
In most of the existing vehicular networks, vehicles and APs employ short-range
wireless communication technologies, current standards of which include IEEE
802.11 and Direct Short Range Communication (DSRC). Channel conditions en-
countered by a vehicular network vary depending on the environment in which
the vehicular network operates. On a highway or in a rural area, it is possible
for vehicles to have line-of-sight communication with other vehicles and road-
side APs. In urban/city areas, road conditions (e.g. straight, curvy) have an
effect on whether line-of-sight communication can be achieved. In addition, the
existence of large number of obstructions like buildings and trees tend to block
54
line-of-sight communication. Further, signals can reflect or deflect on these ob-
structions, causing wireless communication to experience multi-path fast fading
effect and resulting in bursty lossy channel conditions. The velocity of vehicles
also has an impact on channel quality.
Therefore, to have realistic settings for accurate analysis, high-fidelity channel
models that can model the wireless channel conditions of vehicular networks
under different environments should be used. At the current stage, Rayleigh and
Ricean fading models are used in our analysis with the maximum doppler velocity
set based on the vehicle velocity in the mobility traces. Our future work is to
develop/integrate high-fidelity channel models for vehicular networks.
3.2 Phase II: Integrate Network Simulation Into Trans-
portation Simulation
Simulation environments that allow user level traffic simulations and wireless
network simulations to be integrated in a common framework offer substantial
benefits. One key limitation of many existing tools that integrate vehicular traf-
fic and network simulation is the lack of dynamic interactions between the two
domains. Thus transportation simulators would use pre-computed and aggregate
network level delay and packet loss computations whereas network simulators
would use pre-scripted mobility data. The shortcoming of these approaches is
the lack of dynamic interaction between an event (e.g. accident) as it unfolds
in the transportation simulator and its dissemination to the vehicles using the
network as embedded within the vehicles in its vicinity and the feedback to the
transportation simulator the change in velocities and positions of the vehicles as
they react in real-time to the information conveyed to them by the communi-
55
cation network. The lack of the above type of dynamic interaction between the
transportation and network simulator reduces the level of realism that can be
achieved for key applications like active safety and traveler information systems
which in reality influence the vehicles’ movements significantly.
3.2.1 Related Work
The last few years have witnessed a major proliferation of tools that attempt
to integrate vehicular traffic and wireless network simulation [8, 14, 16, 24, 26,
31, 36, 46, 54, 55, 64, 68]. Considering that the motion of vehicles on real street
maps cannot be accurately captured by the commonly used random waypoint
mobility model, one class of these proposals [8, 16, 24, 54] allow users to rapidly
generate realistic VANET mobility traces or models that can be immediately
used by popular network simulators such as ns-2 and QualNet. However this
approach fails to support models of other important traffic components such
as vehicle interaction, intersection actuators, traffic lights, etc. and prohibits
runtime control of vehicle behavior by applications.
The second class of existing approaches incorporate vehicular traffic and wire-
less networking into a single simulation engine [14, 36, 64]. However, due to the
difficulty of writing efficient transportation and wireless simulators from scratch,
the network simulaions tend to lack high-fidelity communication models, vali-
dated VANET protocol models or complex traffic models, as compared with the
existing set of established transportation or network simulators.
The most promising approach is to couple and synchronize existing traffic and
network simulators. The majority of the work in this area [26, 31, 46, 55, 68] has
linked NS-2 or QualNet network simulators with diverse transportation simula-
tors such as SUMO, CARISMA, CORSIM and VISSIM. [68] uses the Federated
56
Distributed Simulation Kit (FDK) runtime infrastructure (RTI) software to in-
tegrate CORSIM, QualNet and applications. In [26], VISSIM incorporates the
module called VCOM that simulates inter-vehicle communication as a dynamic
link library (dll) to allow direct method invocation. The rest of the proposals
in this class [31, 46, 55] use a standard TCP connection to link the transporta-
tion and the network simulators. Our work extends these projects to provide a
platform that can be used to evaluate and compare VANET protocols in terms
of their impact in supporting not just safety applications, but also the emerging
class of VANET applications like ATIS under realistic operating conditions as
determined by both the transportation system and the communication network .
3.2.2 Simulation Platform Architecture and Interfaces
The proposed simulation platform is a composition of two independent simula-
tion packages running in a distributed fashion over multiple networked computers
(see Figure 3.3): VISSIM [62], a microscopic transportation simulator and Qual-
Net [49], a packet-level network simulator. To maintain maximum flexibility and
cross-platform interoperability in the architecture, each simulator is run on in-
dependent machines running different OS, communicating over a TCP socket to
provide a fast and reliable connection. The bi-directional communication be-
tween VISSIM and QualNet enables the run-time control of vehicle behavior.
As VISSIM is distributed as a licensed COTS product with limited source-code,
an external program called VISSIM Control was implemented to (a) perform
the communication tasks to QualNet via TCP sockets and (b) interact with the
VISSIM Component Object Model (COM) interface (discussed below) to control
the transportation simulation as well as to access traffic data and alter vehicle
behavior. The last component in the simulation architecture is the application,
57
Figure 3.3: Simulation platform architecture for the integration of transportation
simulation, network simulation and applications
which provides overall control of the simulation environment. For simplicity,
the VANET applications were implemented in the network simulator, although
emulation frameworks such as those proposed in [72] can be utilized to directly
interface real applications with the network simulator.
3.2.2.1 Simulation Components
Transportation simulator and VISSIM Control VISSIM [62] is a micro-
scopic transportation simulator that simulates vehicle interaction, traffic flow
and congestion. VISSIM uses commonly accepted vehicle and driver behavior
models to represent traffic networks. Extensive geometric and operational data
are required to model a network in VISSIM. Data requirements include location
and distance of links (i.e. freeways and local streets), number of lanes on each link
segment, connectors, source and destination parking lots, signalized intersection
control plans, free flow speeds, traffic composition and flows. VISSIM generates
various metrics including average vehicle speed, average and total travel and de-
58
lay times, average link density etc. VISSIM provides a COM interface which
allows full control to most aspects of a VISSIM simulation, including modifica-
tion of attributes such as speed and route of a vehicle. The proposed simulation
platform utilizes this COM interface to automate tasks in VISSIM by executing
COM commands from the external VISSIM Control program.
Network Simulator and External Interface The widely used network
simulator QualNet [49] is used to model and simulate vehicular networks in-
cluding various protocols at different layers of the protocol stack and wireless
communication. QualNet provides a comprehensive set of network models for
the entire protocol stack from the application to physical layer and includes a
variety of ad hoc and VANET routing protocols. It also includes high-fidelity
wireless models that incorporate physical environment effects (e.g. fading and
shadowing). Extensive performance metrics for a complete understanding of net-
work behavior can be collected including throughput, latency, dropped packets
etc. QualNet provides the external interface API which allows the simulator to
interact with external entities such as other programs or physical devices. In
the simulation platform, an external interface that links QualNet to VISSIM is
implemented which communicates with VISSIM via the TCP connection.
Applications Within the distributed simulation platform of this phase, the
VANET applications have been integrated into the network simulator as addi-
tional modules at the application layer. This simplifies implementation, as the
existing TCP socket can be leveraged for interaction between the application and
VISSIM. In general, it would be possible to implement the application using a
third environment that interacts via separate interfaces with the two simulators;
however, our approach considerably simplified the simulation architecture with-
out compromising the fidelity of the applications that need to be interfaced with
59
Figure 3.4: Communication among transportation simulator, network simulator
and applications (described in Section 3.2.2.2)
.
the simulation environment.
3.2.2.2 Data Exchange
The interaction among VISSIM, QualNet and applications is described in Figure
3.4. The distributed simulation starts with the initialization phases of both the
transportation and the network simulation. Initially, VISSIM transmits the ge-
ographic size of the simulated traffic network, the number of vehicles currently
in the network and their positions to QualNet (Figure 3.4-(5)). Once the simu-
lation starts, mobility data i.e. vehicle position updates including the entering
and exiting of vehicles to and from the traffic network are sent to QualNet and
mapped to mobile nodes in the wireless network simulation.
To represent the behaviors of individual vehicles, an application queries VIS-
SIM about various attributes of the current vehicle (Figure 3.4-(10)). In partic-
60
ular, Dynamic Route Planning requires information including a vehicle’s current
position, speed and the index of the next intersection it is heading towards. Re-
ceiving such queries, VISSIM feedback the relevant traffic data to the application
(Figure 3.4-(11)). Events generated by an application, for example, the occur-
rence of an incident, and the results of computation performed in the application
logic, e.g. a new route to be followed by a vehicle, change the behavior of individ-
ual vehicles by assigning new values to a vehicle’s attributes. Data of such change
of behavior are communicated from applications to VISSIM to be incorporated
into the simulation of the traffic network (Figure 3.4-(12)).
Messages generated by an application to be disseminated into the vehicular
network are transfered to QualNet (Figure 3.4-(13)), which models the handling
of these messages by VANET protocols running at different layers of the network
protocol stack and wireless radio propagation. Successfully received messages are
delivered back to the application for further processing (Figure 3.4-(14)).
3.2.2.3 Synchronization
VISSIM and QualNet must be synchronized such that the shared attributes be-
tween the simulators are kept consistent for accurate modeling of the system.
The common practice in existing integrations of a transportation and a network
simulator [31,46,55] is to use periodic time synchronization, where each simulator
runs autonomously for a pre-specified synchronization horizon (e.g. one second),
at the expiry of which, common attributes are synchronized (e.g. a request for
new mobility data is sent from the network simulator to the transportation sim-
ulator), and the process is repeated periodically every synchronization horizon.
We implement an adaptation of the time-stepped synchronization mechanism
that maintains consistency of shared attributes regardless of the relative execu-
61
tion speed of VISSIM and QualNet. The synchronization mechanism utilizes a
variable, ExternalSimTime, maintained at the external interface connecting
QualNet with VISSIM. As VISSIM is inherently a time-stepped simulator, its
control logic is used to set the duration of the next time step, which is communi-
cated to QualNet and implemented using the ExternalSimTime variable, which
determines how far ahead in the future QualNet can advance before it needs
to interact with VISSIM. Note that although QualNet uses a parallel discrete-
event synchronization algorithm internally, this scheme allows us to separate out
QualNet’s ability to run the network simulator on parallel machines from its in-
teractions with VISSIM. In addition, the duration of the next time step can be
dynamically adjusted during a simulation run based on the required fidelity and
the number of traffic and wireless communication events. Initially, both simula-
tions start at virtual time 0 and run one time step. The simulation that gets to
the next time step first waits for the other to finish. Once both simulations are
at the same time step, QualNet issues a query to VISSIM for mobility updates
and VISSIM transmits the up-to-date mobility data. At this point, VISSIM also
incorporates the received change of behavior data. Afterwards, both simulations
start to (independently) execute the next time step and the above process repeats
for the specified simulation duration.
62
CHAPTER 4
Evaluation of Multihop Relaying for Robust
Vehicular Internet Access
4.1 Introduction
As people continue to spend substantial amount of time in their daily lives trav-
eling using either private vehicles or public transport, their need to stay con-
nected to the Internet and have access to information on the move is becoming
increasingly important. Until recently, cellular networks served as the primary
means for vehicular Internet access. Though the current generation of cellular
networks provides wider coverage, they are plagued by low and variable data
rates (especially at vehicular speeds), high and variable latencies, and occasional
communication blackouts (depending on the mobile node’s spatial location) [50].
Besides, users are also required to subscribe to their data services. With the
widespread deployment of WiFi (802.11) [20] access points (APs) everywhere
and the introduction of DSRC standards to enable intelligent transport systems
(ITS), WiFi-like technologies are becoming a promising alternative for vehicle to
infrastructure/roadside communication (necessary for Internet access) as well as
for inter-vehicular communication. This shift is mainly driven by performance
and cost considerations.
When using WiFi for Internet access in highly dynamic vehicular environ-
63
ments, ensuring continuous and seamless connectivity becomes the primary issue
because of the relatively smaller communication range of WiFi devices (compared
to cellular-based access). While recent measurement studies [7, 12, 43] demon-
strate the viability of WiFi for vehicular Internet access, they also suggest that
such access will be suitable mainly for applications tolerating intermittent con-
nectivity because of the short duration of connections observed (in the order of
few tens of seconds) [7]. These studies, however, only focus on direct commu-
nication between vehicles and roadside APs, and do not consider inter-vehicular
communication.
Internet connectivity in vehicular environments using WiFi devices depends
on several factors, including: AP density and distribution, vehicle density, distri-
bution and speed, and communication range of nodes (APs and vehicles). Some
of these factors may not be easy to influence (e.g., the number of APs and their
locations), whereas others like communication range allow some degree of control.
Though increasing the communication range by using higher transmission power
can extend the coverage, the extent to which this can be done is limited due to
regulatory restrictions and hardware limitations. Moreover, higher transmission
power can reduce overall network throughput due to increase in interference and
reduction in spatial reuse opportunities. Increasing the communication range us-
ing other physical layer modalities also involve similar tradeoffs, such as lowering
transmission bit-rates for longer ranges.
Having vehicles not directly connected to APs depend on other vehicles to
relay packets, possibly over multiple hops using inter-vehicular communication,
offers a seemingly better alternative to improve connectivity as it does not require
high power transmissions nor force the use of lower bit-rates. Such a multihop re-
laying strategy can exploit greater connectivity opportunities resulting from high
64
density of vehicles. Multihop relaying as a design strategy has been found to be
beneficial in other contexts (e.g., improving the coverage and data transfer perfor-
mance of home wireless networks [44] and wireless LANs [30], improving aggregate
and end-user data rates while preserving fairness by using heterogeneous wireless
technologies [34]). Internet connectivity for mobile ad hoc networks (MANETs)
also involves multihop relaying [52]. But none of these past efforts give insight
into the connectivity properties of multihop relaying expected in real-world ve-
hicular environments. On the other hand, there has been considerable amount of
work in vehicular networks involving inter-vehicular communication, focusing on
routing, measurements and such (see [57], for example). But, as far as we know,
this body of work does not consider connectivity issues arising from communica-
tion with fixed infrastructure as is the case with vehicular Internet access. There
has also been some work on analyzing connectivity properties of (hybrid) ad hoc
networks [4, 10], where the focus is on connectivity between nodes in an ad hoc
(multihop wireless) network with or without the use of wired infrastructure. In
contrast, our focus is on connectivity between mobile nodes (vehicles) and the
fixed Internet, possibly via multiple wireless hops.
This chapter presents a case study, the goal of which is to study the potential
connectivity improvement from using multihop relaying via inter-vehicular com-
munication as opposed to relying only on direct communication between APs
and vehicles (referred henceforth as direct access). We also study the effect of
communication range for both strategies; this is in contrast to prior measure-
ment studies [7], which focus only on one extreme setting of radio parameters,
i.e., lowest bit-rate and maximum transmission power. To meet the above goals,
we study spatio-temporal aspects of connectivity for direct access and multihop
relaying strategies by analyzing real AP location data in conjunction with realis-
tic vehicular mobility trace for a city scenario (in our case, we consider the city
65
of Zurich, Switzerland). We conduct this study independent of any specific ve-
hicular network protocols and applications, but focusing on connectivity metrics
like connection duration and percentage of vehicles connected, which are relevant
for supporting any application. Our evaluation approach allows us to efficiently
study connectivity characteristics of large scale vehicular network scenarios with
several thousands of vehicles and hundreds of APs. For instance, we were able
to process mobility traces spanning a four hour period and containing as many
as 4000 vehicles in few tens of minutes.
4.2 Methodology
We consider two communication strategies: direct access and multihop relaying.
Recall that direct access refers to a common communication strategy where a
vehicle is connected to the Internet only when it is in the coverage area of an
AP. This is similar to the WLAN architecture that is commonly used in WiFi
networks. On the other hand, multihop relaying strategy allows vehicles not
directly connected to any AP to depend on other vehicles for relaying their pack-
ets, possibly over multiple hops using inter-vehicular communication. For both
communication strategies, we assume a commonly used strongest signal strength
based AP selection policy to determine the AP a vehicle associates with when
faced with multiple choices. Once a vehicle is associated with an AP, it stays
associated to the same AP until they move out of each other’s communication
range. We further assume that a vehicle remains directly connected as long as it
is in the coverage area of some AP. With multihop relaying, a vehicle not directly
connected to any roadside AP uses a relay path (involving other vehicles) with
least hop count and below a specified hop count threshold, if available. A path
once selected is used as long as it is valid. A vehicle remains connected as long
66
6.8 6.81 6.82 6.83 6.84 6.85x 105
2.42
2.43
2.44
2.45
2.46
2.47
2.48
2.49 x 105
X Coordinate
Y Co
ordi
nate
(a) Distribution of roadside APs
0 1 2 3 40
500
1000
1500
2000
2500
3000
3500
4000
Time (hour)
Num
ber o
f Act
ive
Vehi
cles
(b) Vehicle density over time
Figure 4.1: Spatial distribution of APs and vehicle density variation over time in
a selected region in the city of Zurich, Switzerland.
as it has a path to an AP satisfying the hop count threshold. When studying
connectivity with multihop relaying, we consider the effect of using different hop
count thresholds.
We use the city of Zurich, Switzerland as a representative scenario for our con-
nectivity characterizations. This choice was influenced by the ready availability
of detailed vehicular movement traces for the Zurich region. The mobility trace
is obtained from a detailed vehicular movement simulation over real road maps
using MMTS [40]. It contains a 24-hour movement pattern (coordinates, moving
directions and speeds) of a total number of 259, 978 vehicles in Switzerland (area
of 41, 559km2). The traces are further parsed to obtain movement data for se-
lected regions and scenarios (e.g., city, highway) and with desired vehicle density
and speed. The results presented in this case study correspond to a small region
(28Km2 in area) in Zurich city. We obtained AP location data for this region
from www.jiwire.com. There are a total of 132 APs, whose spatial distribution is
shown in Fig. 4.1(a) with x, y coordinates in Swiss projection coordinate format.
67
We use a subset of the vehicular mobility trace corresponding to this selected
region and a 4-hour rush hour period. Variation of number of vehicles (and ve-
hicle density) during the 4-hour period is shown in Fig. 4.1(b). The minimum,
mean and maximum vehicle speeds in this trace were 1m/s, 16m/s and 33m/s
respectively.
For determining the radio communication range, we make the following as-
sumptions. We assume the 802.11b physical layer and omnidirectional antennas
(placed at 1.5m height). Receiver sensitivity values used for various transmission
rates are shown in Table 4.1. For the channel, we assume two-ray ground reflec-
tion based radio propagation path loss model and constant shadowing with mean
4.0dB. We do not consider the effect of small-scale fading, which does not affect
our observations about the relative merits of multihop relaying and direct access
communication strategies with regard to connectivity. Different communication
range values in our study are obtained from varying transmission power and rate
values. Table 4.2 summarizes the different power and rate combinations used and
associated communication range values. Two power values used were obtained by
looking up typical and maximum power values used in commodity 802.11b wire-
less network interface cards (specifically, the Proxim ORiNOCO Gold 802.11b/g
card).
4.3 Performance Results
This section presents our results studying the impact of communication strat-
egy (i.e., direct access versus multihop relaying) and communication range on
vehicular Internet connectivity. Broadly speaking, we study connectivity across
the spatial and time dimensions. Spatial connectivity at a given time is mea-
sured as the fraction of vehicles connected at that time, whereas connection and
68
Rate Receiver Sensitivity
1Mbps -93dBm
2Mbps -89dBm
5.5Mbps -87dBm
11Mbps -83dBm
Table 4.1: Receiver sensitivity values assumed at different 802.11b transmission
rates.
Rate (⇓), Power (⇒) 15dBm 19dBm
1Mbps 483m 609m
2Mbps 370m 467m
5.5Mbps 353m 445m
11Mbps 283m 357m
Table 4.2: Communication range values for different 802.11b transmission rate
and power level combinations.
69
disconnection durations are used as metrics for temporal connectivity.
Fig. 4.2 shows the benefit of multihop relaying and increased communica-
tion range with respect to spatial connectivity over time, corresponding to the
4-hour period shown in Fig. 4.1(b). Fig. 4.2(b) and 4.2(c) correspond to multihop
relaying with hop count threshold set to 2 hops and 3 hops, respectively. Individ-
ual curves in each plot represent specific communication range values obtained
from various power and rate combinations shown (see Table 4.2). Comparing
Fig. 4.2(a), 4.2(b) and 4.2(c), we observe that multihop relaying gives substantial
improvement in coverage over direct access for the same communication range.
Direct access with increased communication range and multihop relaying seem to
provide similar gains. For instance, compare (19dBm, 1Mbps) curve in Fig. 4.2(a)
to (15dBm, 11Mbps) curve in Fig. 4.2(c). Combination of multihop relaying and
increased communication range provides the best coverage overall. It is also in-
teresting to note that there is no correlation between vehicle density and spatial
connectivity (compare Fig. 4.1(b) and Fig. 4.2). Spatial distribution of vehicles
at different vehicle densities shown in Fig. 4.3 helps explain this behavior and
suggests that increased vehicle density leads to more uniform increase in vehicles
across all road segments. Essentially, given that AP locations are fixed, the num-
ber of connected vehicles proportionately increases with the number of vehicles,
thereby keeping spatial connectivity unaffected by vehicle density variation.
Moving onto temporal connectivity, Fig. 4.4 shows average connection du-
ration1 over time, obtained by averaging across all vehicles in each 250 second
interval. Like in the case of spatial connectivity, multihop relaying fairs better
than direct access, and the combination of multihop relaying and increased com-
munication range gives the most improvement. But, relatively speaking, the use
1Note that our estimate of connection duration is at a coarse level in that it includes over-heads like AP association and IP address acquisition latencies.
70
0 1 2 30
0.2
0.4
0.6
0.8
1
Time (hour)
Frac
tion
of V
ehic
les
Conn
ecte
d
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(a) Direct access
0 1 2 30
0.2
0.4
0.6
0.8
1
Time (hour)
Frac
tion
of V
ehic
les
Conn
ecte
d
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(b) Two-hop relaying
0 1 2 30
0.2
0.4
0.6
0.8
1
Time (hour)
Frac
tion
of V
ehic
les
Conn
ecte
d
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(c) Three-hop relaying
Figure 4.2: Spatial connectivity (fraction of vehicles connected) over time with
direct access and multihop relaying strategies at different power and rate combi-
nations (reflecting different communication range values).
71
6.8 6.81 6.82 6.83 6.84 6.85x 105
2.42
2.44
2.46
2.48
x 105
X Coordinate
Y Co
ordi
nate
(a) At 0.25 hour
6.8 6.81 6.82 6.83 6.84 6.85x 105
2.42
2.44
2.46
2.48
x 105
X Coordinate
Y Co
ordi
nate
(b) At 0.5 hour
6.8 6.81 6.82 6.83 6.84 6.85x 105
2.42
2.44
2.46
2.48
x 105
X Coordinate
Y Co
ordi
nate
(c) At 1 hour
Figure 4.3: Spatial distribution of vehicles in the selected region of Fig. 4.1(a)
after first 15 minutes, 30 minutes and 1 hour in the 4-hour period shown in
Fig. 4.1(b).
72
0 1 2 30
100
200
300
400
500
600
Time (hour)
Aver
age
Conn
ectio
n Du
ratio
n (s
ec)
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(a) Direct access
0 1 2 30
100
200
300
400
500
600
Time (hour)
Aver
age
Conn
ectio
n Du
ratio
n (s
ec)
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(b) Two-hop relaying
0 1 2 30
100
200
300
400
500
600
Time (hour)
Aver
age
Conn
ectio
n Du
ratio
n (s
ec)
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(c) Three-hop relaying
Figure 4.4: Connection duration, averaged across all vehicles over 250 second
time intervals, with direct access and multihop relaying strategies at different
power and rate combinations (reflecting different communication range values).
73
of multihop relaying is more effective than direct access with increased communi-
cation range, especially at higher vehicle densities. This can be explained by the
clustered distribution of APs (see Fig. 4.1(a)) and the ability of multihop relay-
ing strategy to exploit higher vehicle densities for improving connectivity. This
is because AP clustering increases the likelihood of vehicles moving in and out of
their range, which hurts temporary connectivity of direct access with increased
communication range. Multihop relaying, on the other hand, allows using other
vehicles as relays to stay connected. We note that there is noticeable though
smaller gain in connection duration with increased vehicle density even for direct
access as vehicles move slowly at higher densities.
Connection duration statistics (CDF, average and median), taken over all con-
nections across all vehicles over the whole four-hour period, are shown in Fig. 4.5
and Table 4.3. Corresponding data for disconnection duration (contiguous period
without connectivity) are given in Fig. 4.6 and Table 4.4. These results clearly
highlight the value of multihop relaying as an effective and flexible mechanism for
achieving long connectivity periods (close to a factor of two improvement over di-
rect access strategy with increased communication range — from 212.36 seconds
to 376.59 seconds). When seen together with negligible disconnection periods,
multihop relaying with increased communication range makes it feasible to stay
connected most of the time.
We have also investigated the impact of path length on gains with multihop
relaying. First, we studied the path length (hop count) distribution when hop
count threshold is set to infinity (i.e., no limit). Path length CDF (Fig. 4.7)
shows that most paths are only few hops long (at most 3-4 hops), regardless of
the communication range. Table 4.5 studies the percentage of gain in connection
duration with multihop relaying relative to direct access for increasing hop count
74
0 500 1000 1500 20000
0.2
0.4
0.6
0.8
1
Connection Duration (sec)
Frac
tion
of C
onne
ctio
ns
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(a) Direct access
0 500 1000 1500 20000
0.2
0.4
0.6
0.8
1
Connection Duration (sec)
Frac
tion
of C
onne
ctio
ns
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(b) Two-hop relaying
0 500 1000 1500 20000
0.2
0.4
0.6
0.8
1
Connection Duration (sec)
Frac
tion
of C
onne
ctio
ns
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(c) Three-hop relaying
Figure 4.5: Connection duration CDF with direct access and multihop relaying
strategies at various communication range values.
75
0 200 400 6000
0.2
0.4
0.6
0.8
1
Disconnection Duration (sec)
Frac
tion
of D
isco
nnec
tions
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(a) Direct access
0 200 400 6000
0.2
0.4
0.6
0.8
1
Disconnection Duration (sec)
Frac
tion
of D
isco
nnec
tions
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(b) Two-hop relaying
0 200 400 6000
0.2
0.4
0.6
0.8
1
Disconnection Duration (sec)
Frac
tion
of D
isco
nnec
tions
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
(c) Three-hop relaying
Figure 4.6: Disconnection duration CDF with direct access and multihop relaying
strategies at various communication range values.
76
Average (median) 1hop 2hop 3hop
connection duration (s)
(15dBm, 11Mbps) 66.65 (37) 124.45 ( 70) 206.24 (166)
(15dBm, 2Mbps) 98.04 (66) 188.37 (152) 252.49 (210)
(15dBm, 1Mbps) 126.73 (84) 320.26 (250) 367.20 (268)
(19dBm, 1Mbps) 212.36 (195) 372.51 (272) 376.59 (273)
Table 4.3: Average and median connection duration with direct access and mul-
tihop relaying strategies at various communication range values.
Average (median) 1hop 2hop 3hop
disconnection duration (s)
(15dBm, 11Mbps) 61.23 (25) 36.86 (6) 22.42 (1)
(15dBm, 2Mbps) 59.40 (19.34) 29.36 (1) 6.74 (0.69)
(15dBm, 1Mbps) 33.14 (8) 6.33 (0.49) 1.87 (0.38)
(19dBm, 1Mbps) 26.56 (0.91) 2.56 (0.37) 0.49 (0.35)
Table 4.4: Average and median disconnection duration with direct access and
multihop relaying strategies at various communication range values.
77
1 2 3 4 5 60.2
0.4
0.6
0.8
1
Hop CountFr
actio
n of
Pat
hs
15dBm, 11Mbps15dBm, 2Mbps15dBm, 1Mbps19dBm, 1Mbps
Figure 4.7: Path length CDF at various communication range values for multihop
relaying with hop count threshold set to infinity.
thresholds. We observe that going from direct access (threshold = 1) to two-hop
relaying yields the highest improvement, with further increase in the threshold
giving diminishing returns, which suggests that few hops are sufficient to get most
of the gain with multihop relaying. We also observe that communication range
influences the gain from increased hop count threshold.
4.4 Discussion
The foregoing results suggest that multihop relaying is a promising strategy for
vehicular environments from the connectivity viewpoint. Even though our eval-
uations are based on AP location data and vehicular mobility trace for one city
(Zurich, Switzerland), we expect our results to hold generally for two reasons.
First, AP density and distributional characteristics (e.g., clustering) observed in
our study seem to match that of data reported for other cities [1,7]. Second, ve-
hicular mobility traces we used are not based on real data for a specific city, but
instead obtained via realistic microscopic vehicular movement simulation that is
78
% gain over 2hop 3hop 4hop 5hop
direct access
(15dBm, 11Mbps) 86.72 209.44 285.51 400.53
(15dBm, 2Mbps) 92.14 157.54 248.34 251.51
(15dBm, 1Mbps) 152.71 189.75 195.94 196.49
(19dBm, 1Mbps) 75.41 77.34 77.95 77.97
Table 4.5: Percentage of gain in connection duration with multihop relaying rela-
tive to direct access for increasing hop count thresholds at various communication
range values.
likely to be applicable more generally.
A crucial next step to assess if multihop relaying strategy improves Internet
access in vehicular environments is understanding its data transfer performance.
Multiple access interference and channel dynamics (fading) are key factors in this
regard. While the capacity scaling issues of multihop wireless networks are well
known, it is unclear whether relay assisted vehicular Internet access networks with
small diameter are also interference limited. Besides, conducting realistic data
transfer performance evaluation in this setting is closely tied to the protocols and
techniques used (e.g., routing, channel allocation, link adaptation and mobility
management), whereas the latter task of designing vehicular Internet access pro-
tocols is challenging in itself (e.g., performing route maintenance seamlessly and
efficiently). Also cross-layer approach may be needed for best performance. For
instance, a key observation from our study is that multihop relaying combined
with increased communication range is the most effective strategy to achieve
seamless and continuous connectivity, which suggests the need for a cross-layer
approach involving at least network and link layers for achieving robust vehicular
79
Internet access. In our on-going and future work, we plan to address the above
issues.
Let us turn our attention to the implications of clustered AP distribution
(as observed in our study, see Fig. 4.1(a)) on vehicular Internet access protocol
design. During our analysis of connectivity characteristics, we have noticed that
only 40% of the APs are used by the vehicles for association because of clustered
AP distribution. With large number of vehicles, each AP then may have to serve
up to as many as 80 vehicles, resulting in overloading of a small fraction of the
APs. The above observation points to the need for intelligent AP selection and
association schemes that take AP load into consideration. Another consequence of
clustered AP distribution is that AP coverage areas tend to overlap considerably.
This in turn makes seamless connectivity provisioning easier through the use of
smooth handoff techniques (e.g., concurrent association of vehicles with multiple
APs).
4.5 Summary
We have studied the connectivity benefits of enabling a multihop relaying strategy
via inter-vehicular communication for WiFi-based vehicular Internet access. A
unique aspect of our study is the use of real AP location data and detailed vehic-
ular movement traces. Overall, our results show that multihop relaying strategy
leads to substantial gains in connectivity, and that multihop relaying combined
with increased communication range provides even greater gains. We also found
that relay paths with few hops are sufficient to realize most of the gain with mul-
tihop relaying. The focus of our on-going and future work is on understanding
the data transfer performance with multihop relaying and on developing suite of
effective protocols for enabling robust vehicular Internet access.
80
CHAPTER 5
Evaluation of Video Streaming over Vehicular
Networks
In this chapter, the utility and benefits of the high-fidelity application-centric
evaluation framework proposed in Section 2.4 and 3.1 are demonstrated through
a case study. The case study investigates the performance of a video streaming
application running in a vehicular network scenario. The studied scenario in-
cludes three different sub-networks: the Internet, the vehicular network and the
wireless LAN within a vehicle. It is shown in the case study how the proposed
framework can be used to realize an appropriate model for complicated vehicular
network scenarios like this. As running operational applications is enabled by
the evaluation framework, the performance of video streaming is studied using
application-level metrics such as PSNR.
There are two key points that we hope to demonstrate by the case study.
• Show the utility and value of the application-centric evaluation framework
for vehicular networks. By using TWINE, real applications specific to ve-
hicular networks can be executed directly such that network performance
can be measured at the application layer using application-specific metrics.
In this way, the extent to which the quality of service of various classes
of applications (e.g. safety applications, commercial services) provided by
the current generation of vehicular networks can be investigated effectively.
81
In our study, by comparing the Peak-Signal-to-Noise-Ratio (PSNR) of a
video streaming application delivered by two routing protocols (AODV and
GPSR) and their respective network-level performance, we were able to
show that application-level metrics are more directly related to end user
experience and thus provide more reliable performance results. The study
also highlights scenarios where network-level statistics including through-
put, delay and jitter fail to discriminate between the two routing protocols
while significant performance differences were observed using application-
level metrics, i.e. order of tens of dB on PSNR improvement and a 38.3%
reduction on mean square root of error achieved by AODV over GPSR.
• The proposed framework facilitates the investigation of cross-layer interac-
tions across the protocol stack which includes applications. This is demon-
strated in our study by examining the effects of the interaction among video
streaming application, two network-layer routing protocols and a MAC-
layer rate adaptation protocol. The evaluation framework enables the study
of possible correlation between application-level and network-level perfor-
mance, which is crucial in identifying appropriate network-level metrics that
are capable of accurately reflecting application layer performance.
5.1 Experiment Setup
The experiments conducted in this study investigate the performance of a video
streaming application running in a vehicular network. The network scenario has
vehicles moving on a freeway which are equipped with wireless devices and thus
able to communicate with each other (refer to Figure 5.1). Along the freeway
there are APs that serve as gateways to the Internet. The vehicles can connect
82
Internet
RoadsideBase-station
Client
Vehicle Router
Vehicular Network
Figure 5.1: Vehicular network scenario for video streaming
with these APs, possibly over multi-hop routes, to access the Internet. A wireless
LAN that connects various devices such as laptops, PDAs, portable media players
and gaming consoles inside a vehicle. The application used is a media player that
displays, on a client device in a vehicle, a streaming video from a server on the
Internet.
Three different sub-networks are included in this scenario: the Internet, the
vehicular network, and the wireless LAN. The proposed application-centric eval-
uation paradigm is used to realize an appropriate model for this scenario: The
WLAN includes the client running a legacy video streaming application, so this
network (clients and AP) is modeled as an emulated network, running the op-
erational client application. The Internet is also operational since legacy video
streaming servers are used. The vehicular network is, however, simulated. Thus
the network comprises of physical, simulated and emulated components and the
traffic flows across all these modes.
A MacBook Pro is used as the client machine which runs VLC [63] streaming
video client application. The streaming server runs VLC in server mode. For
83
the routing protocol, AODV [45] and GPSR [25] are used, as they represent two
large classes of ad hoc routing protocols: reactive non-geographic and geographic
with greedy forwarding. Both protocols are well documented, tested in many
research studies and shown to exhibit excellent performance in their respective
class of routing protocols. For MAC and PHY, IEEE 802.11b is used, in which
the APs and vehicles maintain a fixed transmission rate at 11Mbps, given the
relatively high bandwidth requirement of video. The channel effects are modeled
using two-ray pathloss model and Rayleigh fading model with varied maximum
fading velocity.
At the start of each experiment, the client is located next to an AP so that
it can communicate with the AP directly. As time advances, the vehicle drives
away from the AP and needs to communicate through multiple hops. A default
setting used to configure the experiments has 45 vehicles on the highway, moving
in both directions at an average speed of 30±2m/s. A single vehicle serves as the
client and receives the streaming video from a roadside AP. The video is encoded
at 112Kbps. Rayleigh fading is turned off in the default setting to isolate the
effects of fast fading.
5.2 Performance Results
5.2.1 Demonstrate Utility and Benefits of Application-Centric Eval-
uation Paradigm
To effectively illustrate the advantages associated with the application-centric
evaluation framework, as compared to the traditional network-centric evaluation,
Figure 5.2 plots the network-level performance of GPSR and AODV in terms of
throughput, delay, jitter and loss. It is observed that across all the metrics, the
84
0 10 20 30 40 500
200
400
600
800
Time(sec)
Thro
ughp
ut (K
bps)
GPSRAODV
(a) Throughput
0 10 20 30 40 500
2
4
6
8
10
12
Time (sec)De
lay
(sec
)
GPSRAODV
(b) Delay
0 10 20 30 40 50!2
!1
0
1
2
Time (sec)
Jitte
r
GPSRAODV
(c) Jitter
0 10 20 30 40 500
200
400
600
800
Time (sec)
Loss
GPSRAODV
(d) Loss
Figure 5.2: Network-level performance of GPSR and AODV in terms of through-
put, delay, jitter and loss respectively in default setting
85
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(a) GPSR
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(b) AODV
Figure 5.3: Application-level performance of GPSR and AODV in terms of PSNR
in default setting
relative performance of GPSR and AODV varies over time. It is, however, hard
to discriminate between GPSR and AODV in terms of the overall performance.
Further, due to the lack of a thorough understanding of the correlation between
application-level and network-level performance, given these results, it is not
straightforward to estimate the streaming video quality to be expected by an end
user.
In contrast, the application-level performance of GPSR and AODV is shown
in Figure 5.3 and Table 5.1. Figure 5.3 plots the PSNR for each video frame
received at the client. The maximum value of PSNR, which corresponds to no
distortion of video, is set to be 100dB. The shaded areas in the figure can be
viewed as the time instances when the video is corrupted and the magnitude
indicates the extent of corruption. Seen in the figure, it is obvious that overall,
AODV delivers higher quality video compared to GPSR, especially towards the
end of the experiment when the video is streamed through multi-hop routes from
the roadside to the vehicle client. By looking at these figures, one can easily
86
Application-level Metric GPSR AODV
Root Mean Square of Error 5.22 3.22
Corrupted Frames 18.44% 10.16%
Number of Corrupted Intervals 10.40 8.00
Avg Duration of Corrupted Intervals (sec) 0.70 0.43
Corrupted blocks per minute 14.02 11.13
Corrupted sec per minute of video 9.18 4.79
Avg PSNR in corrupted intervals (dB) 21.70 21.55
Table 5.1: Application-level performance of GPSR and AODV in terms of other
application-layer metrics in default setting
discern how the video application is performing over time.
Such temporal results can be aggregated into quantities that are listed in
Table 5.1. Each quantity shown is an average of multiple runs of the same exper-
iment configuration. Root mean square of error indicates the average distortion
over all the received video frames. It is noted that the video delivered by GPSR
has higher average distortion than AODV, which is consistent with the PSNR re-
sults. Metrics including corrupted frames, number of corrupted intervals, average
duration of corrupted intervals, corrupted blocks per minute and corrupted sec
per minute of video reflect the degree of distortion of the received video and the
distribution of corrupted frames over time. To summarize the results on these
metrics, the use of GPSR results in more corrupted frames and longer corrupted
intervals. The last metric, average PSNR of corrupted intervals, represents how
badly the frames are corrupted. It is interesting to see that although the overall
performance of AODV is better than GPSR, in terms of corrupted frames, GPSR
produces slightly higher quality.
87
The results shown in the above figures and table demonstrate that application-
level metrics are more directly related to end user experience and therefore facil-
itate the understanding of the quality of service of applications achieved by the
operation of a target vehicular network. Another point illustrated by these re-
sults is that there exist cases where network-level performance represented by the
commonly-used network-level metrics bears little correlation with appli-cation-
level performance, i.e. such network-level metrics fail to discriminate between the
two routing protocols while significant performance difference can be observed at
the application layer. In general, the correlation of network-level and application-
level performance relates closely to the specific nature and implementation de-
tails of the application. Depending on the application, it can be complicated
to identify the appropriate set of network-level metrics that closely reflect the
performance at the application layer. The potential relationship of network-level
and application-level performance requires a significant amount of effort to be
studied. This provides another argument to use application-level metrics in net-
work performance analysis to expedite the evaluation process and obtain reliable
results. In our case, by comparing the set of application-level metrics measured
for video streaming, it is observed that PSNR constitutes the best candidate to
effectively reflect the overall user-peceived video quality over time. In the rest of
the chapter, PSNR is used as the primary application-level metric.
To further illustrate the benefits of the evaluation framework, the impact of
vehicular network environment on the application-level performance is studied.
The two parameters used to vary the environment settings are wireless channel
quality (i.e. maximum Ray-leigh fading velocity) and vehicle density. Figure 5.4
plots the PSNR performance of GPSR and AODV with the maximum Rayleigh
fading velocity set to 30m/s, which represents the realistic vehicle speed on a
high way of fair vehicle density. It is seen that fading has a great impact on the
88
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(a) GPSR
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(b) AODV
Figure 5.4: PSNR of GPSR and AODV with Rayleigh fading at max velocity
30m/s
performance of both protocols; neither GPSR nor AODV is capable of delivering
video of fair quality when channel conditions become unfavorable. Such obser-
vation leads to the following implications. First, it is difficult to maintain high
quality video when vehicles are moving very fast on the free way (as compared to
a city scenario). Second, in order to provide video streaming services to vehicles
on a highway, intelligent cross-layer design decisions should be made in order to
efficiently respond to the changing channel.
The impact of vehicle density on the delivered PSNR performance of GPSR
and AODV is illustrated in Figure 5.5. The video is encoded at 256Kbps in this
set of results. First it is noted that the performance of GPSR (see Figure 5.5(a),
5.5(c), 5.5(e)) deteriorates greatly when vehicle density grows. Such behavior can
be explained by the fact that when there exist more vehicles in the network, the
possibility of GPSR choosing an unstable long link increases. Correspondingly,
the perceived video quality drops by a large extent. In contrast, it is seen that
AODV exhibits more resilience to the change in vehicle density (see Figure 5.5(b),
89
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(a) GPSR, 23 vehicles
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(b) AODV, 23 vehicles
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(c) GPSR, 45 vehicles
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(d) AODV, 45 vehicles
0 10 20 300
20
40
60
80
100
Time(sec)
PSNR
(dB)
(e) GPSR, 91 vehicles
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(f) AODV, 91 vehicles
Figure 5.5: PSNR of GPSR and AODV at different vehicle densities with video
rate 256Kbps
90
5.5(d), 5.5(f)). The performance of AODV remains rather constant at various
densities. These results show that the nature (positive or negative) and the extent
of the impact on the application-level performance by vehicle density depend
on the particular algorithm a routing protocol adopts. PSNR is effective in
demonstrating such impact at the application layer. In both cases, however, the
rich connectivity provided by a large number of vehicles existing in the network
does not help in improving the delivered video quality. This emphasizes that
when designing a protocol, not only the possible impact of vehicular network
environment should be taken into account but also the protocol should try to
utilize the knowledge of the environment to further enhance its performance.
5.2.2 Evaluate Application Design and Implementation in the Target
Vehicular Network
As stated in the previous section, application-level performance is closely re-
lated to the implementation details of an application including application-layer
adaptation, optimization, parameter configuration, etc. Through the proposed
application-centric evaluation framework, the investigation of application design
and implementation choices can be performed within the very context of the tar-
get vehicular network, instead of as a stand-alone process. In this section, the
effects of using different video coding rates on the application-level performance
of GPSR and AODV are studied. The results provide a basis to the future study
of adaptive video streaming applications in a vehicular network. Figure 5.6 plots
the PSNR performance of GPSR and AODV at two video rates, 56Kbps and
256Kbps. Together with Figure 5.3, it is clear that AODV outperforms GPSR in
delivering higher quality video – the performance difference enlarges as the video
rate increases. Such application-level performance discrepancy can be traced to
91
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(a) GPSR, 56Kbps video
0 10 20 30 400
20
40
60
80
100
Time(sec)PS
NR (d
B)(b) AODV, 56Kbps video
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(c) GPSR, 256Kbps video
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(d) AODV, 256Kbps video
Figure 5.6: PSNR of GPSR and AODV at different video rates with fixed trans-
mission rate 11Mbps
92
Application-level Metric GPSR AODV
Root Mean Square of Error 10.41 1.41
Corrupted Frames 28.85% 3.69%
Number of Corrupted Intervals 12.40 3.20
Avg Duration of Corrupted Intervals (sec) 0.40 0.57
Corrupted blocks per minute 17.42 4.16
Corrupted sec per minute of video 7.17 2.22
Avg PSNR in corrupted intervals (dB) 20.14 19.06 dB
Table 5.2: Application-level performance of GPSR and AODV with ARF at video
rate 112Kbps
the different loss performance of the two protocols. This implies that adaptation
at the application layer should take into account the properties of the underlying
protocols, in this case the loss behavior of a routing protocol.
5.2.3 Investigate Cross-Layer Interaction across Protocol Stack In-
cluding Applications
The previous study shows the impact of network operation on the achieved user
satisfaction and the need to study cross-layer interaction. These observations are
further corroborated in this section by investigating the effects of the interactions
among the video streaming application, the two routing protocols (GPSR and
AODV) and a MAC-layer rate adaptation protocol (Auto Rate Fallback – ARF)
on the user-perceived video quality. Figure 5.7 plots the PSNR of GPSR and
AODV with ARF at three video rates. Comparing to Figure 5.3 and 5.6, it
is observed that the performance of AODV is enhanced by ARF at video rates
56Kbps and 112Kbps; while at 256Kbps, the performance of AODV deteriorates
93
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(a) GPSR+ARF, 56Kbps
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(b) AODV+ARF, 56Kbps
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(c) GPSR+ARF, 112Kbps
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(d) AODV+ARF, 112Kbps
0 10 20 300
20
40
60
80
100
Time(sec)
PSNR
(dB)
(e) GPSR+ARF, 256Kbps
0 10 20 30 400
20
40
60
80
100
Time(sec)
PSNR
(dB)
(f) AODV+ARF, 256Kbps
Figure 5.7: PSNR of GPSR and AODV with ARF at different video rates
94
0 10 20 30 40 500
200
400
600
800
1000
1200
Time(sec)
Thro
ughp
ut (K
bps)
GPSRAODV
(a) Throughput
0 10 20 30 40 500
2
4
6
8
10
12
Time (sec)De
lay
(sec
)
GPSRAODV
(b) Delay
0 10 20 30 40 50!2
!1
0
1
2
Time (sec)
Jitte
r
GPSRAODV
(c) Jitter
0 10 20 30 40 500
400
800
1200
1600
Time (sec)
Loss
GPSRAODV
(d) Loss
Figure 5.8: Network-level performance of GPSR and AODV with ARF at video
rate 256Kbps
95
largely with the existence of ARF. In contrast, the performance of GPSR drops
significantly at all video rates when ARF is used. Such relative behavior is
also observed on other application-level performance measures, i.e. comparing
Table 5.2 to Table 5.1. These observations indicate that user-perceived video
quality is the result of combined functioning of various protocols at different
layers from the application down to PHY. When different protocols are used
and their configurations vary, application-level performance changes. Further,
the impact of a particular protocol on the application-level performance differs
depending on the choice of protocols at other layers. For instance, at the same
video rate of 112Kbps, the performance of AODV is improved by ARF while the
performance of GPSR impaired.
To understand the reason why the use of ARF is able to cause PSNR to drop,
especially at high video rates, Figure 5.8 shows the network-level performance of
GPSR and AODV with ARF at video rate 256Kbps. It can be seen that ARF
improves delay and jitter, comparing to the case where a fixed transmission rate
of 11Mbps is used, because ARF reduces the transmission rate when it sees packet
drops. However, it is also observed that once ARF decreases the transmission
rate, it stays at that lower rate for a relatively long period of time until sufficiently
large number of successful transmissions are seen. This can be illustrated by the
throughput performance of AODV (refer to Figure 5.8(a)), which stays at three
levels as time advances, corresponding to the three higher transmission rates of
802.11b. Such an approach of ARF to trade throughput for reliability works
effectively with low-rate videos, as the throughput requirement of the application
can be well satisfied even at low transmission rates. However, when the video rate
is high, using low transmission rates fails to meet the application requirement.
As seen in Figure 5.8(d), a large number of packets are dropped. Due to high
loss and low throughput, when video rate is high, e.g. 256Kbps, the application-
96
level performance decreases with the use of ARF. These observations indicate
that a lower layer protocol should be aware of the application requirements in
determining its adaptation strategy. When the application requirements change,
the protocol should adjust its behavior responsively. ARF ignores the application
requirements and reduces transmission rate for higher reliability even when high
throughput is demanded, resulting in its poor performance with high-rate videos.
5.2.4 Explore Correlation Between Application-level and Network-
level Performance
Another research direction opened up by the application-centric evaluation frame-
work is to study the correlation between application-level and network-level per-
formance. In Figure 5.3, it is seen that the performance difference between AODV
and GPSR becomes more prominent during the last 10 seconds of the experiment.
Associating the PSNR performance with the network-level performance (see Fig-
ure 5.2), it is noted that as the network-level performance drops, the perceived
video quality decreases correspondingly. This conforms with the general observa-
tion that in order to deliver high quality video, high throughput, low delay, low
jitter and low loss are required. It is observed that with AODV, video frames
are more likely to be dropped in bursts, as opposed to randomly with GPSR.
Such behavior stems from the specific path selection algorithm used by the two
routing protocols. In MPEG-1, a video clip consists of key frames that encode
all the information of the current frame, and delta frames which only store the
incremental change of the current frame from other relevant ones. When a delta
frame is lost, the subsequent delta frames will apply the delta function using an
incorrect frame as reference. If these frames are lost at regular intervals, such
as in the case of GPSR, the errors may accumulate to show significant deviation
97
in the received video from the original one. The above discussion indicates that
with the current set of network-level metrics, it is hard to represent the corre-
lation between application-level and network-level performance in a quantitative
manner. A new set of metrics, which examine the network-level performance in
more detail than the first order, are required to be designed in order to effectively
study the impact of network operation on end user experience.
5.3 Summary
The quantitative data provided by the set of experiments presented in this chap-
ter show that the evaluation framework proposed in Section 2.4 and 3.1 is able to
deliver more accurate and reliable results compared to the traditional network-
centric evaluation. Further, shown by case studies, it opens up future research
directions including but not limited to the evaluation of application design and
implementation in the context of the target vehicular network environment,
the study of cross-layer interaction and the investigation of correlation between
application-level and network-level performance. It is our hope that the pro-
posed evaluation framework offers a suitable platform to advance the research in
vehicular networks in the fields outlined above.
98
CHAPTER 6
Evaluation of VANET-based Advanced
Intelligent Transportation Systems
A major contribution of this chapter derives from the observation that most stud-
ies of VANET systems focus on safety applications such as emergency warning
and are limited to measuring network characteristics like packet delivery ratio
and latency. Very few results have been published that relate these network met-
rics to relevant metrics at the application or transportation system level. In this
chapter, we present case studies that evaluate the performance of a representative
Intelligent Transportation Systems (ITS) application – Dynamic Route Planning,
using metrics collected at the transportation system level (e.g., travel time, delay
time, vehicle density and speed, traffic volume, traffic knowledge availability and
accuracy).
The case studies demonstrate how VANETs can significantly expand the de-
ployment of Advanced Traveler Information Systems (ATIS). The application in
our case study – Dynamic Route Planning – is a specific kind of ATIS where in-
dividual vehicles react to current and changing traffic conditions to make better
routing decisions so that travel time is reduced and trip reliability increased. Ex-
isting ATIS rely only on road-side sensors whose deployment costs limit their ap-
plicability to major highways. However, by linking ATIS with VANETs, vehicles
can share travel time information in a peer-to-peer fashion, significantly enhanc-
99
ing their applicability without incurring corresponding high cost of widespread
roadside sensor deployments. An issue with this approach is that since the wire-
less channel is shared by every vehicle in VANET, the channel can be easily
saturated due to the transmission of traffic information when vehicle density is
high. Hence, the load imposed by such exchange of data needs to be carefully
controlled for reliable and low-latency transmission. Various schemes have been
proposed to save the bandwidth consumed by traffic data transmission. We con-
duct case studies to compare three representative algorithm/protocols, cost-based
and hierarchical aggregation, adaptive broadcast, and distributed fair power ad-
justment (D-FPAV) that aim to reduce the traffic data load. The case studies are
performed within the distributed vehicular network simulation platform proposed
in Section 3.2, which enables run-time change of behavior of vehicles’ movements
in the transportation simulation as a result of application-layer events computed
by Dynamic Route Planning based on the results of information exchange via
wireless communication among vehicles. Once again, we show the impact of
these alternatives not just by comparing traditional network level metrics, but
in terms of how they impact the traffic system as represented by transportation
simulators like VISSIM.
6.1 Dynamic Route Planning
In our implementation of Dynamic Route Planning, each vehicle maintains a
knowledge base of vehicle and travel time information which it uses to dynami-
cally improve its route. The knowledge base consists of two types of records: (1)
vehicle record, containing fields of IDveh, Position, Speed, Direction and Record
T ime, that provides updated information for multiple vehicles and (2) travel time
record, containing fields of IDseg, Travel T ime and Record T ime, which provides
100
updated information on the state of multiple road segments that the vehicle is
expected to traverse. Note that Record T ime is the global time when the in-
formation about a vehicle or road segment is originally recorded. With a GPS
receiver and a digital map installed, a vehicle is able to keep track of its location
and speed. When a vehicle reaches the end of an identified road segment, IDseg,
it records its travel time. Each vehicle periodically broadcasts information in its
knowledge base. Whenever a vehicle receives a broadcast message, it incorporates
the data contained in the message into its knowledge base. When the next broad-
cast period comes, the vehicle broadcasts the updated information. By sharing
traffic information in such a peer-to-peer fashion, vehicles can be aware of the
traffic situation of the entire road network and accordingly change their routes
to avoid congested areas. Each vehicle maintains an estimated travel time for
every road segment in the network to dynamically compute the optimal route.
Initially, when a vehicle enters into the network, the estimated travel time is set
to be the free flow travel time, i.e. the time to travel through a road segment at
its speed limit. The estimated travel time is subsequently updated either when a
vehicle records its own experienced travel time or whenever a more recent travel
time record of the corresponding road segment is received.
6.1.1 Route Computation
From the experiment results, it is seen that the response time to congestion, i.e.
delay from the time instance the congestion happens till the time instance vehicles
start to travel on alternative routes to avoid the congested area, is relatively long
(explanation given in Section 6.4.2). To improve performance, a supplement
method to compute the estimated travel time is implemented, where each vehicle
maintains a second attribute for every road segment – estimated travel speed.
101
This estimated travel speed is calculated as the weighted average of the current
estimated value and a newly reported one. Equation 6.1 shows the computation
of weight coefficients, where T is the current simulation time, (ve, te) the current
estimated travel speed and record time, and (v, t) the speed and record time of
the received record. The new estimated travel speed and record time (v′e, t
′e) are
computed as in Equation 6.2 and 6.3 respectively. Subsequently, the estimated
travel time is calculated by dividing the distance of the road segment by its
estimated travel speed.
αe =T − t
(T − te) + (T − t)α =
T − te(T − te) + (T − t)
(6.1)
v′
e = ve × αe + v × α (6.2)
t′
e = T − ((T − te)× αe + (T − t)× α) (6.3)
These two types of information (i.e. vehicle speed and travel time) are chosen
to be handled in different manners in the computation of the estimated travel
time because a reported vehicle speed is considered to be relatively random while
a reported travel time more stable. Comparing the record time of the estimated
travel time computed using the two methods, a vehicle sets the final estimated
travel time to the one with more recent record time.
6.1.2 Information Aging
Information aging is implemented to eliminate obsolete or inaccurate records.
Whenever a record is received, a module called Receive Aging [9] calculates the
expected latency for receiving the record and compares that to the actual latency.
102
If the difference between these two is lower than a threshold, the record is merged
into the knowledge base; otherwise, it is considered out-of-date and ignored. In
addition, a timer is associated with each record in the knowledge base. This
timer is reset each time the record is updated by a more recent one. If the
timer expires, the record is dropped. When there are multiple records containing
information about the same vehicle or road segment (in the broadcast message
and the knowledge base), only the most recent record is kept and older versions
removed.
6.2 VANET Protocols
As discussed earlier in the chapter, an issue with deploying Dynamic Route Plan-
ning on VANET is that the wireless channel shared by vehicles can be easily
saturated due to the transmission of traffic data. The load imposed by traffic
data can be controlled by adjusting three primary parameters: (1) broadcast
message size (2) broadcast interval and (3) transmission power (correspondingly,
transmission range). In this case study, we present performance comparisons
on the effectiveness of a representative protocol that implements each of these
approaches.
6.2.1 Aggregation
If the spatial density of vehicles is assumed to be approximately constant, the
amount of traffic data increases quadratically with the covered radius, increasing
size of data to be broadcast by each vehicle thus limiting system scalability. To
overcome this problem, the use of data aggregation has been proposed: with in-
creasing distance, observations concerning larger and larger areas are combined
103
into one single value. Coarse aggregates are made available at greater distances,
more detailed data are kept only in the near vicinity. In this case study, Cost-
Based Aggregation [9] is implemented to aggregate vehicle records and Hierar-
chical Aggregation [32] to aggregate travel time records.
6.2.2 Adaptive Broadcast
Traffic data load on the wireless channel can also be controlled by adjusting the
inter-transmission interval of broadcast messages. In this case study, Adaptive
Broadcast proposed in [67] is implemented. The basic idea of Adaptive Broad-
cast is as follows. Upon the reception of a broadcast message P , a weight wP is
computed based on the comparison of the vehicle and travel time records con-
tained in P with the local knowledge base. For each record ri in P , i = 1, ..., n,
let roi be the corresponding record of the same vehicle or road segment in the
knowledge base. If the difference between the record time of ri and roi exceeds
the time threshold ∆T , wP is increased by a constant qtime (so-called time quan-
tum). Similarly, if the difference of information values (e.g. speed or travel time)
of the two records exceeds the information threshold ∆I, wP is increased by qinfo
(so-called info quantum). Therefore, a message is assigned a high weight if it
contains significantly different information (including both info and time differ-
ence). In contrast, a low weight means this vehicle has a very similar view of the
traffic network as the vehicle that has sent the message. wP is then compared
to the thresholds winc and wdec to determine how the remaining time until the
next broadcast transmission should be adjusted. wP being less than winc causes
an increase of the remaining time; and wP larger than wdec decreases this time.
104
6.2.3 Distributed Fair Power Adjustment
Transmission power used by a vehicle to send out broadcast messages can be
adjusted dynamically in response to vehicle density observed in the network so
as to save bandwidth consumed by traffic data. Each vehicle sends a broadcast
message with a certain transmission power p ∈ [Pmin, Pmax]. The function of
a transmission power control scheme is to compute a power assignment A that
assigns to every vehicle vi, with i = 1, ..., n, a ratio Ai ∈ [0, 1]. The power used
by vehicle vi is Pmin + Ai × (Pmax − Pmin). The objective to be achieved is that
using A, the minimum of the transmission powers used by vehicles for broadcast
is maximized, and the network load experienced at the vehicles remains below
the predefined threshold Maximum Load (ML). This case study implements
Distributed Fair Power Adjustment for Vehicular Networks (D-FPAV) proposed
in [58]. In D-FPAV, a vehicle continuously collects information about the status
of all the vehicles within its maximum carrier sensing range CSmax. Based on
this information, the vehicle locally computes the maximum common value Pi of
the transmission powers for all the vehicles in CSmax such that the condition on
the ML is not violated.
6.3 Experiment Setup
6.3.1 Scenario and Parameters
The traffic network scenario analyzed in this case study is an area of size 3.5×1.6
km2, consisting of a freeway and the local streets in the surrounding area. The
freeway serves as the main route vehicles take to travel from a designated source
location to a designated destination one. The local streets either intersect with
the freeway (i.e. having entrances or exits for getting on and off the freeway) or
105
run in parallel. These local streets can serve as alternative routes for vehicles when
the freeway is congested. When the simulation first starts, every vehicle travels
on the freeway as this is the fastest route to reach the destination. About three
minutes into the simulation, an accident happens on the freeway at a location half
way to the destination. The immediate vicinity of the accident starts to become
congested causing nearby vehicles to slow down. These vehicles disseminate such
change to other vehicles in the network via broadcast messages, which accordingly
update the estimated travel time. As a result, the freeway is no longer the fasted
route to the destination and vehicles start to use the local streets to bypass the
congested area. Six minutes or so, the congested area starts to clear and finally
opens up at the twelfth minute. The entire simulation is 900 seconds. Besides
the vehicles in question which travel from the source to the destination, there are
other vehicles distributed on the local streets moving with a random source and
destination and disseminating traffic information about the local streets. The
vehicles in question however constitute the major fraction of traffic.
IEEE 802.11b is used as the underlying wireless communication technology for
vehicle broadcast with transmission rate 2Mbps. Two-Ray pathloss model and
Rayleigh fading model are used in the simulation. The default transmission power
is set to be 15dBm. For transmission power control, the range of transmission
powers is [0.0, 20.0]dBm, with each level being 0.5dBm apart. The maximum size
of broadcast messages is 1500bytes. The broadcast rate is chosen from 1, 5 and
10 packets/sec.
6.3.2 Performance Measures
Table 6.1 summarizes the set of metrics used in the case study. Both appli-
cation/system and network level metrics are studied. The definition of most
106
Application and System Level Metrics
Number of Vehicles Arrived
Route Quality Travel Time (s)
Delay Time (s)
Traffic Knowledge Estimation Error (m)
Availability & Accuracy Knowledge Percentage (%)
Density (veh/km)
Congestion Speed (m/s)
Volume (veh/h)
Network Level Metrics
Packet Transmission Throughput (Mbps)
End-to-End Delay (s)
Table 6.1: Performance measures for Dynamic Route Planning
metrics in the table are straightforward. For the couple of less used metrics, they
are defined below.
Delay Time determines, compared to the ideal travel time (no other vehicles,
no signal control), the time delay resulted from the actual travel time of a vehicle.
Knowledge Percentage is a metric used to measure the availability of informa-
tion about other vehicles at a vehicle. The road around each vehicle is divided
into regions of 200 meters long. For each region, the percentage of vehicles in that
region about which the current vehicle knows is defined as the knowledge per-
centage of that vehicle for that region. The knowledge percentage graph presents
the knowledge percentage for each region, averaged over all the vehicle during a
simulation run.
Estimation Error determines the accuracy of information available at a ve-
107
hicle. The road around each vehicle is divided into regions of 200 meters long
and the average error in estimating the positions of vehicles in each region is
calculated. In the accuracy graph, the average error for each region is shown,
averaged over all the vehicles during the simulation run.
6.4 Performance Results
6.4.1 Integration of Network Simulation
The first set of experiments demonstrate the impact of incorporating accurate
network simulations into transportation simulators like VISSIM. We run two ex-
periments with the same scenario setting and parameters (max broadcast message
size 1500bytes, broadcast rate 10pkt/s, transmission power 15dBm and trans-
mission rate 2Mbps). The first experiment is run without QualNet, with the
application integrated directly with VISSIM. Due to the lack of any radio prop-
agation and channel access model, ideal transmission conditions are assumed in
this experiment, i.e. unlimited bandwidth, no transmission delay and no packet
loss. The second experiment is run using an identical scenario, with the QualNet
network simulator used to model realistic VANET operations. Our expectation
is that due to high vehicle density and broadcast rate, the wireless channel will
be saturated, which causes a large fraction of broadcast messages to collide. As a
consequence, vehicles have no means of receiving accurate traffic information and
therefore fail to change their routes to avoid the congested area, which impacts
the transportation system metrics.
The results shown in Figure 6.1, 6.2 and 6.3 confirm our hypothesis. Figure
6.1(a) demonstrates that VISSIM Only predicts that on most part of the freeway,
vehicle density remains constantly low over the entire simulation; only the vicinity
108
0 500 10001500200025003000
10200
400600
8000
30
60
90
120
150
Position on Link (m)
Vissim Only, freeway
Simulation Time (s)
Veh
icle
s D
ensi
ty (
veh/
km)
(a) Vissim Only: only immediate vicinity of
accident location gets congested
0 500 10001500200025003000
10200
400600
8000
30
60
90
120
150
Position on Link (m)
Vissim + Qualnet, freeway
Simulation Time (s)
Veh
icle
Den
sity
(ve
h/km
)
(b) Vissim + Qualnet: entire freeway up to
accident area is congested
Figure 6.1: Vehicle density on the freeway over time, Vissim Only vs Vissim +
Qualnet
of the accident location observes high density. This means that most vehicles are
able to make better routing decisions and take the local streets before reaching
the accident area. While in reality, as shown in Figure 6.1(b), most vehicles get
stuck on the freeway. The same disparity in application performance is shown in
Figure 6.2 concerning the average vehicle density, speed and volume at various
locations on the freeway. Compared to VISSIM Only, for the part of the freeway
leading to the accident area, VISSIM + QualNet produces vehicle density up to
388.9% higher, speed up to 58.3% lower and volume up to 40.5% lower.
The average travel quality experienced by vehicles that have successfully
reached the destination over the course of the simulation is presented in Fig-
ure 6.3. As was the case with other metrics already discussed, the travel quality
exhibited with VISSIM Only and VISSIM + QualNet diverges largely. Twice as
many vehicles appear to have completed their trip with VISSIM Only as com-
pared with VISSIM + QualNet (see Figure 6.3(a)). In addition, for much of the
duration of the accident, VISSIM + QualNet shows that the number of vehicles
109
0 500 1000 1500 2000 2500 30000
20
40
60
80
100
120
accident area
Distance (m)
Avg
Veh
icle
Den
sity
(ve
h/km
)
Vissim OnlyVissim + Qualnet
(a) Average vehicle density
0 500 1000 1500 2000 2500 30000
5
10
15
20
25
30
accident area
Distance (m)A
vg V
ehic
le S
peed
(m
/s)
Vissim OnlyVissim + Qualnet
(b) Average vehicle speed
0 500 1000 1500 2000 2500 30000
500
1000
1500
2000
2500
accident area
Distance (m)
Avg
Vol
ume
(veh
/h)
Vissim OnlyVissim + Qualnet
(c) Average traffic volume
Figure 6.2: Traffic condition on the freeway, Vissim Only vs Vissim + Qualnet
110
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Num
ber
of V
ehic
les
Arr
ived
Vissim OnlyVissim + Qualnet
(a) Number of vehicles arrived
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Avg
Tra
vel T
ime
(s)
Vissim OnlyVissim + Qualnet
(b) Average travel time
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Avg
Del
ay T
ime
(s)
Vissim OnlyVissim + Qualnet
(c) Average delay time
Figure 6.3: Travel quality experienced by vehicles over time, Vissim Only vs
Vissim + Qualnet
that have reached the destination remains constant. In contrast, with VISSIM
Only, the corresponding number of vehicles increases steadily, up to 116.8% higher
than with VISSIM + QualNet. The same effect is seen with the average travel
time metric (Figure 6.3(b)), where VISSIM Only predicts a gradual increase as
vehicles are successfully diverted onto local streets. However, as seen with the
results from VISSIM + QualNet, this metric increases sharply over the duration
of the accident (up to 54% longer) because relatively few vehicles have learned
about the congestion under the realistic operating conditions of the VANET net-
111
0 200 400 600 8000
1
2
3
4
5
congestion duration
Simulation Time (s)
Avg
Thr
ough
put (
Mbp
s)
Vissim OnlyVissim + Qualnet
(a) Average throughput
0 200 400 600 8000
5
10
15
20
congestion duration
Simulation Time (s)
Avg
End
−to
−E
nd D
elay
(s)
Vissim OnlyVissim + Qualnet
(b) Average end-to-end delay
Figure 6.4: End-to-end performance of broadcast transmission, Vissim Only vs
Vissim + Qualnet
work. As a result, the average travel time keeps increasing sharply till the end of
the simulation. The average delay time anticipated by VISSIM Only and VIS-
SIM + QualNet also differs largely, with the performance difference up to 125.8%
(Figure 6.3(c)).
To fully grasp the performance discrepancy caused by the lack of high-fidelity
wireless communication simulation, throughput and end-to-end delay of broad-
cast transmission with VISSIM Only and VISSIM + QualNet are compared in
Figure 6.4. It is seen that under the ideal transmission conditions, the throughput
achieved with VISSIM Only is close to 4Mbps, which indicates that the traffic
data load requires a channel capacity of 4Mbps or higher. With the actual ca-
pacity being only 2Mbps (achievable goodput at the application layer even less),
the throughput is barely 0.2Mbps (Figure 6.4(a)). Similarly, as expected with
VISSIM Only, broadcast transmission experiences zero delay while the realistic
end-to-end delay increases almost linearly the entire duration of congestion with
a maximum delay of 20 seconds (Figure 6.4(b)).
112
In summary, in order to produce performance results that accurately reflect
the behavior of applications and the VANET network under realistic operating
conditions, it is vital to incorporate high-fidelity communication simulations.
6.4.2 Vehicle Information Benefits Route Planning
Various proposals on dynamic route planning (e.g. [32]) use travel time collected
by vehicles as the only source of traffic information for computing routes. After
examining this approach, we observe that the achieved response time to con-
gestion i.e. delay from the time instance the congestion happens till the time
instance vehicles start to travel on alternative routes to avoid the congested area,
is relatively long. This observation is shown in Figure 6.5(a). It is seen that nearly
6 minutes after the accident happens, vehicles start to arrive at the destination
again. The response time to congestion is around three and a half minutes. The
reason of such performance is that as change in traffic condition is solely contained
in travel time information, such dynamic can only be observed and disseminated
when a vehicle reaches the end of the congested road segment and records its
travel time. As this duration directly increases in the presence of congestion,
the lack of timely updates to other vehicles that may be at some distance from
the congested area can worsen congestion. As presented in Section ??, vehicle
information can also be utilized to aid route computation. Figure 6.5 shows that
the use of vehicle information instead of travel time information improves the
performance in terms of the number of vehicles reaching the destination, the av-
erage travel time and the average delay time. The response time to congestion is
only a few seconds, which can be explained by the fact that variation in vehicle
speed can be recorded and disseminated to other vehicles rapidly and efficiently.
The hybrid approach of combing both types of information obviously has the best
113
performance, with up to 121.9% increase on the number of vehicles reaching the
destination, 27% reduction on travel time and 46.9% reduction on delay time.
An interesting observation is that during congestion, the hybrid approach only
performs slightly better than vehicle information only. As soon as the congestion
ends, the performance gain achieved by the hybrid approach increases by a fair
amount. Such behavior relates to how the estimated travel time is updated differ-
ently using the two types of information in our proposed hybrid approach. With
a travel time record, the estimated travel time is simply replaced by the more
recent value. While with vehicle records, the estimated travel speed is computed
as a weighted average, which reflects the relatively long-term change instead of
an instantaneous one. Therefore the estimated travel time determined by travel
time information captures the current traffic condition more accurately, which
results in the performance gain of the hybrid approach in uncongested scenario.
In summary, the inclusion of vehicle information in dynamic route computa-
tion improves the application performance by a fair amount, especially during a
traffic jam. Travel time information contributes to the performance gain effec-
tively in uncongested conditions.
114
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Num
ber
of V
ehic
les
Arr
ived
Traveltime info onlyVeh info onlyTraveltime + Veh info
(a) Number of vehicles arrived
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Avg
Tra
vel T
ime
(s)
Traveltime info onlyVeh info onlyTraveltime + Veh info
(b) Average travel time
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Avg
Del
ay T
ime
(s)
Traveltime info onlyVeh info onlyTraveltime + Veh info
(c) Average delay time
Figure 6.5: Travel quality over time, travel time information only, vehicle infor-
mation only vs combination of both
6.4.3 Effectiveness of Aggregation
The impact on the performance of Dynamic Route Planning of aggregating traffic
data to be disseminated is shown in Figure 6.6, with broadcast message size of
500 and 1500 bytes. In the case of heavy data load imposed on the channel due
to large packet size, e.g. 1500 bytes, travel quality experienced by vehicles such
as travel time is slightly improved by the use of aggregation that provides rela-
tively broader knowledge of the traffic network for a vehicle to compute routes
115
(Figure 6.6(a)). However, as a large fraction of data exchange is collided, the
knowledge percentage achieved is nevertheless fairly low although aggregation is
applied (Figure 6.6(b)). This results in insufficient improvement on travel quality
by aggregation. When the message size is reduced to 500 bytes, the benefit of
conveying more information by aggregation becomes prominent, achieving up to
20% increase on knowledge percentage, in particular about regions further away.
Travel time in this case, however, is longer with aggregation (Figure 6.6(a)). This
is related to the fact that Dynamic Route Planning favors accurate information
about its immediate vicinity in order to make the optimal decision as to which
road segment to take the next time instant; for larger areas, it requires only
coarse information to determine the general direction. Aggregation although im-
proves knowledge availability, sacrifices information accuracy, therefore impaires
the application performance in this case (Figure 6.6(c)).
In summary, the use of aggregation improves the performance of Dynamic
Route Planning when channel is congested. Reducing the message size while ap-
plying aggregation however does not help with the application performance due
to insufficient data accuracy. For applications that have less stringent require-
ments of accuracy but prefer wider knowledge of the traffic network, aggregation
may be a more effective approach.
6.4.4 Effectiveness of Adaptive Broadcast
This set of experiments evaluate the effectiveness of Adaptive Broadcast in sup-
porting Dynamic Route Planning. Table 6.2 describes the different parameter
sets used for the parameters introduced in Section 6.2.2. The main parameters
varied over the four sets are the info quantum qinfo, the time quantum qtime
and the info threshold ∆I. From the discussion in Section 6.2.2, these parame-
116
200 400 600 8000
100
200
300
400Message size=1500B
Simulation Time (s)
Avg
Tra
vel T
ime
(s)
200 400 600 8000
100
200
300
400Message size=500B
Simulation Time (s)
Avg
Tra
vel T
ime
(s)
No aggregationAggregation
No aggregationAggregation
(a) Average travel time
0 500 1000 1500 20000
20406080
100Message size = 1500B
Distance (m)Kno
wle
dge
Per
cent
age
(%)
0 500 1000 1500 20000
20406080
100Message size = 500B
Distance (m)Kno
wle
dge
Per
cent
age
(%)
No aggregationAggregation
No aggregationAggregation
(b) Knowledge percentage
0 500 1000 1500 2000 2500 3000 35000
500
1000
1500Message size = 1500B
Distance (m)
Est
imat
ion
Err
or (
m)
0 500 1000 1500 2000 2500 3000 35000
500
1000
1500Message size = 500B
Distance (m)
Est
imat
ion
Err
or (
m)
No aggregationAggregation
No aggregationAggregation
(c) Average estimation error
Figure 6.6: Travel quality and availability and accuracy of traffic knowledge,
aggregation vs no aggregation
117
ters, in combination with ∆T , winc and wdec, determine how fast the adaptation
of inter-transmission interval is performed. Figure 6.7 shows the travel quality
achieved by Adaptive Broadcast with these different parameter sets, compared to
the scheme of using a fixed transmission rate of 10pkt/s. The corresponding aver-
age inter-transmission interval at each vehicle as a result of adaptation is plotted
in Figure 6.8. Adaptive Broadcast is seen to have the best performance with set
2 in which case the inter-transmission time is adjusted aggressively by adding a
large quantum to the weight whenever a record is received containing sufficiently
different information (performance gain of up to 118% on the number of vehicles
reaching the destination, 36.2% on travel time and 56.1% on delay time). This,
in contrast to the deficient performance of Adaptive Broadcast with set 1 infers
that Dynamic Route Planning requires accurate traffic information, therefore fre-
quent data exchange among vehicles. Another observation made from the poor
performance of Adaptive Broadcast with set 3 is that info difference alone is not
a reliable enough source for inter-transmission interval adaptation decisions. The
performance, however, can be largely improved by having a small info thresh-
old as in set 4, which increases the sensitivity of the inter-transmission interval
adaptation to info difference.
In summary, Adaptive Broadcast has the potential to significantly enhance the
performance of Dynamic Route Planning. This however is achieved given that
the parameters are carefully set to tailor the requirements of the application.
The final parameter values relate to the type of traffic information used by the
application and the degree of sensitivity the application’s performance to the
variation in such information.
118
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Num
ber
of V
ehic
les
Arr
ived
10pkt/sAB, para1AB, para2AB, para3AB, para4
(a) Number of vehicles arrived
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Avg
Tra
vel T
ime
(s)
10pkt/sAB, para1AB, para2AB, para3AB, para4
(b) Average travel time
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Avg
Del
ay T
ime
(s)
10pkt/sAB, para1AB, para2AB, para3AB, para4
(c) Average delay time
Figure 6.7: Travel quality experienced by vehicles over time, fixed broadcast rate
vs Adaptive Broadcast
119
Parameter Set 1 Set 2 Set 3 Set 4
qinfo 0.001 0.005 0.005 0.005
qtime 0.001 0.005 0.000 0.000
∆I (m/s) 4.5 1
∆T (s) 60
winc 0.01
wdec 0.04
Table 6.2: Protocol Parameter Values for Adaptive Broadcast
200 400 600 8000
5
10
15
20
25
Vehicle ID
Avg
Bro
adca
st in
terv
al (
s)
AB, para1AB, para2AB, para3AB, para4
Figure 6.8: Average inter-transmission interval at each vehicle with four different
protocol parameter sets of Adaptive Broadcast
120
6.4.5 Effectiveness of D-FPAV
The effectiveness of the power control protocol D-FPAV in supporting Dynamic
Route Planning is studied by the set of experiments presented in this section. Fig-
ure 6.9 shows the travel time achieved by D-FPAV with Maximum Load threshold
(ML) set at three different values (i.e. 1, 1.5 and 2 Mbps), in comparison to the
scheme of fixed transmission power of 15dBm. The transmission rate used in
all the cases is 2Mbps. It is noted that the value set for ML threshold affects
the performance of D-FPAV. In this scenario, D-FPAV performs the best with a
more strict threshold of 1Mbps (up to 21.9% reduction on travel time). Overes-
timating the capacity of the channel by having a large ML indeed impairs the
performance. However, as D-FPAV performs nearly as poorly with ML being
1.5Mbps, overestimation is unlikely the main cause of such performance. The
knowledge percentage about the traffic network maintained at each vehicle is
plotted in Figure 6.10. It is shown that due to high collisions of data trans-
mission, vehicles have fairly incomplete view of the traffic network. For regions
within the maximum transmission range (452.492m), the knowledge percentage
drops below 50%. The accuracy of D-FPAV heavily depends on the knowledge a
vehicle has about other vehicles in at least one maximum carrier sensing range.
With insufficient knowledge, D-FPAV underestimates the contention for the chan-
nel from other vehicles in the network when it computes the estimated network
load and thus assigns overly high transmission power to vehicles. Figure 6.11
confirms this by comparing the transmission power that would be assigned to
a vehicle with perfect knowledge (D-FPAV Perfect) and the actual transmission
power used. With a tight ML, the difference to the ideal transmission power is
reduced, resulting in better performance of D-FPAV.
In summary, although D-FPAV is designed to control transmission load im-
121
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Avg
Tra
vel T
ime
(s)
15dBmD−FPAV,ml=2MbpsD−FPAV,ml=1.5MbpsD−FPAV,ml=1Mbps
Figure 6.9: Travel time, fixed transmission power vs D-FPAV
0 500 1000 1500 20000
20
40
60
80
100
Distance (m)
Kno
wle
dge
Per
cent
age
(%)
D−FPAV,ml=2MbpsD−FPAV,ml=1.5MbpsD−FPAV,ml=1Mbps
Figure 6.10: Knowledge percentage of traffic network, D-FPAV
122
0 200 400 600 8000
5
10
15
20
Simulation Time (s)
Tra
nsm
issi
on P
ower
(dB
m)
D−FPAV,ml=2MbpsD−FPAV Perfect,ml=2MbpsD−FPAV,ml=1MbpsD−FPAV Perfect,ml=1Mbps
Figure 6.11: Distribution of computed transmission power, D-FPAV vs perfect
knowledge
posed on the channel in scenarios where the channel is congested as a result
of high vehicle density, the performance of D-FPAV depends on vehicles having
fairly accurate knowledge about the traffic network, which is difficult to achieve
with an overloaded channel. With a strict ML condition, D-FPAV is able to
improve the application performance. However, if ML is set offline and cannot
be dynamically adjusted, when the network becomes less congested, this may
result in unnecessarily low transmission power assigned to vehicles and in turn
the waste of channel bandwidth.
6.4.6 Relative Performance of Protocols
The set of experiments presented in this section compare the performance of
Adaptive Broadcast, D-FPAV and combined Adaptive Broadcast + D-FPAV,
with the scheme of fixed broadcast rate and fixed transmission power of 10pkt/s,
15dBm as the base line. The travel quality achieved by these schemes is plot-
ted in Figure 6.12. It is shown that Adaptive Broadcast is the most effective in
improving the performance of Dynamic Route Planning. The hybrid approach
123
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Num
ber
of V
ehic
les
Arr
ived
10pkt/s,15dBmAdaptive Broadcast,15dBm10pkt/s,DFPAVAdaptive Broadcast+DFPAV
(a) Number of vehicles arrived
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Avg
Tra
vel T
ime
(s)
10pkt/s,15dBmAdaptive Broadcast,15dBm10pkt/s,DFPAVAdaptive Broadcast+DFPAV
(b) Average travel time
200 400 600 8000
100
200
300
400
congestion duration
Simulation Time (s)
Avg
Del
ay T
ime
(s)
10pkt/s,15dBmAdaptive Broadcast,15dBm10pkt/s,DFPAVAdaptive Broadcast+DFPAV
(c) Average delay time
Figure 6.12: Travel quality experienced by vehicles over time, comparing adaption
at different layers
improves the performance of D-FPAV but fails to outperform Adaptive Broad-
cast. This indicates that the adaptation performed at the application layer plays
the dominant role in determining the final performance of the application.
6.4.7 Impact of Penetration Ratio
This set of experiments study the impact of penetration ratio on the improve-
ment of travel quality achieved by Dynamic Route Planning. Penetration ratio
124
is defined to be the fraction of vehicles in the traffic network that are equipped
with wireless communication devices and participate in information exchange.
In all of the previous experiments, penetration ratio is assumed to be 100%,
i.e. every single vehicle in the network collects and transmits traffic information.
In this context, tipping point denotes the penetration ratio beyond which the
performance gain of Dynamic Route Planning becomes marginal. We first run
experiments with realistic traffic network setting where the freeway has three
lanes while the local street only one (i.e. freeway has larger capacity than lo-
cal streets). Figure 6.13(a) plots the gain on the number of vehicles reaching
the destination with penetration ratio ranging from 10% to 100%, compared to
the case of penetration ratio being zero. It is seen that in this scenario, tipping
point is 50%, meaning 50% penetration ratio is sufficient for Dynamic Route
Planning to achieve significant performance gain e.g. 65% more vehicles are able
to arrive at the destination. Beyond this point, the additional gain obtained is
negligible, possibly becoming unworthy of the cost of equipping a larger fraction
of vehicles with computing and communication capabilites. Next we run the
same experiment with the freeway having one lane as well, representing the case
where alternative routes have comparable capacity as the main route. The gain
achieved by Dynamic Route Planning at various penetration ratio in this scenario
is plotted in Figure 6.13(b). It is seen that the same amount of gain achieved
at the tipping point in the three-lane freeway case can be achieved at a smaller
penetration ratio of 30% in this scenario. As long as there is sufficient capacity
on alternative routes, the performance gain keeps increasing. The tipping point
in this scenario becomes 80%.
In addition to relative capacity between alternative routes, another factor
that affects the value of tipping point is channel bandwidth. Figure 6.13(c)
shows the gain on the number of vehicles reaching the destination in the three-
125
0 100 200 300 400 500 600 700 800 9000
50
100
150
200
250
congestion duration
Bandwidth = 11Mbps, freeway = 3 lane, local = 1 lane
Simulation Time (s)
Gai
n on
Num
ber o
f Veh
icles
Arri
ved
10%30%50%70%90%100%
(a) 3 lane freeway, Bandwidth = 11Mbps
0 100 200 300 400 500 600 700 800 9000
50
100
150
200
250
congestion duration
Bandwidth = 11Mbps, freeway = 1 lane, local = 1 lane
Simulation Time (s)G
ain
on N
umbe
r of V
ehicl
es A
rrive
d
10%30%50%70%90%100%
(b) 1 lane freeway, Bandwidth = 11Mbps
0 100 200 300 400 500 600 700 800 9000
50
100
150
200
250
congestion duration
Bandwidth = 2Mbps, freeway = 3 lane, local = 1 lane
Simulation Time (s)
Gai
n on
Num
ber o
f Veh
icles
Arri
ved
10%30%50%70%90%100%
(c) 3 lane freeway, Bandwidth = 2Mbps
Figure 6.13: Gain on vehicles reaching the destination, compared to penetration
ratio = 0%
126
lane freeway scenario but with limited channel bandwidth of 2Mbps. First it is
noted that the tipping point decreases to 30% due to channel saturation. Beyond
tipping point, the performance gain decreases greatly. This is different from
the case of sufficient channel bandwidth where the performance gain at higher
penetration ratio only becomes insignificant but never decreases. This means that
channel saturation greatly impairs the effectiveness of Dynamic Route Planning.
Therefore, penetration ratio needs to be carefully controlled in the case of limited
channel bandwidth.
6.5 Summary
Using the developed simulation platform for vehicular networks, case studies are
performed to investigate the feasibility and performance limitations of VANETs
in support of Dynamic Route Planning. The performance results show that the
incorporation of accurate network simulations into transportation simulators like
VISSIM (refer to Section 3.2) is crucial in producing reliable performance results
of applications operating under realistic conditions of VANET networks. The per-
formance discrepancy caused by the lack of high-fidelity wireless communication
simulation can be up to 116.8% in the number of vehicles reaching the destina-
tion, 54% in travel time and 125.8% in delay time. The case study shows that
Dynamic Route Planning can be effectively supported by VANETs. Among all
the adaptation protocols, Adaptive Broadcast is the most effective in enhancing
the performance of Dynamic Route Planning.
127
CHAPTER 7
Conclusions
This chapter provides a concluding overview of the implications of this research.
First, Section 7.1 discusses the contributions of this dissertation. Succinctly, this
dissertation is an effort that aims to shift the paradigm of wireless network perfor-
mance evaluation from the conventional network-centric perspective to a perspec-
tive that focuses on application-level and system-level performances, which more
truthfully reflect the experience of end users of an networking system. Guided
by such an objective, this dissertation has first developed and presented an eval-
uation framework which enables application-centric performance evaluation and
facilitates accurate and efficient studies of vehicular networks. Using this evalu-
ation framework, the effectiveness of vehicular networks in supporting emerging
applications and services of transportation systems as well as the performance of
various protocols and optimizations in enhancing the operation of vehicular net-
works are evaluated from the prospective of end user experience i.e. the perceived
quality of applications by drivers and passengers on the road. Section 7.2.3 de-
scribes how the simulation framework and the obtained performance results and
observations can be further extended and applied to future performance studies
and protocols designs.
128
7.1 Contributions
The main contributions of this dissertation include two parts: (1) the devel-
opment of a high-fidelity application-centric evaluation framework for vehicular
networks and (2) performance evaluation conducted within this framework of
various vehicular network architectures, applications and protocols. The devel-
opment of the evaluation framework is accomplished in two steps. At the first
step, an application-centric evaluation paradigm is proposed which, by utilizing a
hybrid emulation testbed, addresses the unique challenges of vehicular networks
and enables the evaluation of vehicular networks from the application-centric
perspective. Real applications specific to vehicular networks can be executed di-
rectly such that network performance can be measured at the application layer
using application-specific metrics. In this way, the extent to which the quality
of service of various classes of applications (e.g. safety applications, ITS, com-
mercial services) provided by the current generation of vehicular networks can
be investigated effectively. Guided by this evaluation paradigm, the second step
further extends the capabilities of the evaluation framework by realizing a dis-
tributed simulation platform that allows runtime control of vehicle behavior in
the transportation simulation by events generated by an application as a result of
information exchange in the communication network. The simulation platform is
composed of a microscopic transportation simulator VISSIM and a packet-level
network simulator QualNet, running independently on different machines, linked
dynamically via a standard TCP connection for fast and reliable communication.
The proposed simulation platform facilitates the measurement of performance
metrics at the application and transportation system level.
The evaluation framework is then used to examine the feasibility and perfor-
mance limitations of vehicular networks in supporting various classes of applica-
129
tions including traditional Internet applications like video streaming, file transfer
and web browsing, and advanced intelligent transportation systems such as Dy-
namic Route Planning. The first case study is devoted to investigate the potential
connectivity improvement from using multihop relaying via inter-vehicular com-
munication as opposed to relying only on direct communication between APs and
vehicles. The study leads to the following key observations:
• Multihop relaying provides substantial gains in connectivity relative to di-
rect access, with small number of relays sufficient to achieve most of this
gain. The additional gain in connectivity from allowing additional hops
tends to diminish after a few hops, and this gain is dependent on the com-
munication range. For the considered scenarios, going from direct access
to two hop relaying provides the highest improvement in most cases (up to
152%).
• In terms of spatial connectivity (measured as percentage of vehicles con-
nected), multihop relaying and direct access with increased communication
range yield similar improvements. Combination of multihop relaying and
increased communication range provides the most gain (up to 150%).
• With regard to temporal connectivity metrics (connection and disconnec-
tion durations), multihop relaying provides greater improvement compared
to direct access with increased communication range. As with spatial con-
nectivity, multihop relaying with increased communication range is the most
effective strategy, which achieves gain as much as 467%.
• Spatial connectivity is unaffected by vehicle density, whereas connection
duration improves with higher vehicle density.
• The implications of AP distributional characteristics for vehicular Internet
130
access protocol design are also discussed. In particular, the clustered AP
distributions observed in the case study not only necessitate intelligent AP
selection policies capable of load balancing among nearby APs, but also
create opportunities for performing seamless handoffs.
The second case study compares the Peak-Signal-to-Noise-Ratio (PSNR) of
a video streaming application delivered by two routing protocols (AODV and
GPSR) and their respective network-level performance. The case study shows
that application-level metrics are more directly related to end user experience
and thus provide more reliable performance results. The study also highlights
scenarios where network-level statistics including throughput, delay and jitter fail
to discriminate between the two routing protocols while significant performance
differences were observed using application-level metrics, i.e. order of tens of
dB on PSNR improvement and a 38.3% reduction on mean square root of error
achieved by AODV over GPSR. The results from this study lead to the following
insights.
• The adaptation at the application layer should take into account the prop-
erties of the underlying protocols, in this case the loss behavior of a routing
protocol.
• The impact of a particular protocol on the application-level performance
differs depending on the choice of protocols at other layers.
• A lower layer protocol should be aware of the application requirements in
determining its adaptation strategy. When the application requirements
change, the protocol should adjust its behavior responsively.
• A new set of metrics, which examine the network-level performance in more
detail than the first order, are required to be designed in order to effectively
131
study the impact of network operation on end user experience.
Using the distributed simulation platform, the third case study investigates
the feasibility and performance limitations of VANETs in support of Dynamic
Route Planning. The highlights of the experiment results from the case studies
are summarized as follows.
• Incorporating accurate network simulations into transportation simulators
is proven to have significant impact on the predicted performance of appli-
cations under realistic operating conditions of the VANET network. The
performance discrepancy caused by the lack of high-fidelity wireless commu-
nication simulation is shown to be up to 116.8% in the number of vehicles
arrived at the destination, 54% in travel time, 125.8% in delay time, 388.9%
in vehicle density, 166.9% in vehicle speed and 71.1% in traffic volume.
• With calibrated parameter values, Adaptive Broadcast substantially en-
hances the performance of Dynamic Route Planning, by up to 118% on
the number of vehicles arrived at the destination, 36.2% on travel time and
56.1% on delay time.
• The effectiveness of D-FPAV is reduced due to incomplete knowledge about
the traffic network maintained at vehicles (less than 50% for regions within
one transmission range) in scenarios of high vehicle density. By setting
a strict maximum load threshold, D-FPAV is able to achieve performance
gain of up to 21.9% on travel time.
• The penetration ratio required for Dynamic Route Planning to achieve suf-
ficient performance gain in scenarios of high channel bandwidth is deter-
mined by the relative roadway capacity on alternative routes. In these
132
cases, tipping point increases as the relative capacity enlarges. With lim-
ited channel bandwidth, higher penetration ratio beyond the tipping point
greatly impairs the application performance due to channel saturation.
7.2 Future Work
7.2.1 Extend Distributed Simulation Platform to Execute Real Ap-
plications
As discussed in Section 3.2, at the current stage of development of the distributed
simulation platform for vehicular networks, VANET applications have been inte-
grated into the network simulator as additional modules at the application layer.
In order to facilitate the measurement of application-level metrics for complex
applications such as video streaming and tele-conferencing, the capabilities of
the simulation platform should be enhanced such that the running of operational
application softwares is enabled. One possible approach is to utilize the hybrid
emulation test bed TWINE [72] to interface real applications with the network
simulator, as proposed in Section 2.4. In general, it would be possible to imple-
ment the application using a third environment that interacts through separate
interfaces with the transportation and the network simulators.
7.2.2 Examine Interaction of Periodic and Event-driven Messages
The second piece of future work aims at extending the case study presented
in Chapter 6. In vehicular networks, in addition to periodic broadcast mes-
sages, so-called “beacons”, proactively sent out by vehicles, when a hazardous
situation is detected, “reactive” or “event-driven” emergency messages are trans-
mitted. Example applications of this class are emergency warning systems and
133
road-condition warning systems. Messages generated by such safety applications
should be granted priority over periodic beacons. Since the wireless channel is
shared by every vehicle in VANET, without careful control of vehicles’ communi-
cation behavior, the channel can be easily saturated due to the transmissions of
beacons. Hence, the load on wireless channel imposed by beacons need to be con-
trolled in order to allow for reliable and low-latency transmissions of high-priority
emergency messages. The case study can be extended by
• Examining the effectiveness of dynamic adaptation schemes in restraining
overload of beacons and thus enhancing QoS provided to event-driven mes-
sages
• Investigating the possible interaction between the two classes of applications
when they co-exist in a vehicular network system.
• Examining the effectiveness of possible QoS approaches in handling multiple
classes of services in vehicular networks.
For the generation of event-driven messages, Accident Alert can be selected
as an example of emergency safety applications. Accident Alert refers to the
class of safety applications in which warning messages are broadcasted in the
event of an accident to notify vehicles in the proximity so that they can make
informed decisions about which route to take in order to bypass the accident area.
By avoiding the accident area, vehicles attempt to minimize possible travel delay
caused by the accident and as a result prevent possible congestion at the accident
area. An alert message contains the time and location of the accident. Such
information reaches vehicles in the surrounding area through periodic broadcasts
of the alert by vehicles that have received such messages. Upon receiving an
alert, based on the imbedded information and inferred traffic situation, a vehicle
134
Application-Specific Metrics
Travel time (s)
Travel Quality Delay time (s)
Queue time (s)
Vehicles reaching destination (%)
Density (veh/m)
Congestion Speed (m/s)
Volume (veh/s)
Queue length
Table 7.1: Application-specific metrics for Accident Alert
determines whether to change its current route and if so, which route to follow.
Table 7.1 summarizes the set of metrics used in the experiments to evaluate the
performance of Accident Alert. Metrics including travel time, delay time, queue
time and vehicles reaching the destination measure the travel quality of vehicles as
a result of accident notification. The rest of the metrics including density, speed,
volume and queue length are chosen to represent the congestion conditions in the
vicinity of the accident area. Most of the metrics are defined in the same way as
in the case study of Dynamic Route Planning. Queue length is the number of
vehicles counted from the location of the accident upstream to the final vehicle
that is in congestion (queue) condition.
7.2.3 Design Adaptive Protocol to Control Traffic Data Load
The last piece of future work is an effort to apply the insights obtained from
the case study presented in Chapter 6 into designing an adaptive protocol which
controls the data load imposed on the wireless channel by traffic information ex-
135
change while optimizing the performance of the application, e.g. Dynamic Route
Planning. The results from this case study show that the effectiveness of such
a protocol is dependent on the contention conditions of the channel. Due to
this reason, a channel estimation mechanism is included in the protocol which
estimates the degree of saturation of the channel using (1) knowledge of vehicle
distribution within the distance of two maximum carrier sensing range and (2)
statistics of past packet transmission. Based on the estimated contention situa-
tion, the protocol determines the appropriate adaptation strategies, for example,
increasing inter-transmission interval or reducing transmission power. The pro-
tocol is also application-aware, i.e. knowing the nature and the quality of service
requirements of the application, so that optimal values for protocol parameters
can be chosen to maximize the application performance. For example, in times
of traffic congestion, Dynamic Route Planning favors accurate traffic information
and therefore aggressive broadcast rate adaptation to produce timely updates.
136
References
[1] A. Akella, G. Judd, S. Seshan, and P. Steenkiste. Self-Management inChaotic Wireless Deployments. In Proceedings of 11th Annual InternationalConference on Mobile Computing and Networking (MobiCom’05), 2005.
[2] M. M. Artimy, W. Robertson, and W. J. Phillips. Assignment of DynamicTransmission Range Based on Estimation of Vehicle Density. In Proceed-ings of 2nd ACM International Workshop on Vehicular Ad Hoc Networks(VANET’05), 2005.
[3] P. Barford and M. Crovella. Generating Representative Web Workloadsfor Network and Server Performance Evaluation. In Proceedings of ACMJoint International Conference on Measurement and Modeling of ComputerSystems (SIGMETRICS’98), 1998.
[4] C. Bettstetter. On the Minimum Node Degree and Connectivity of a Wire-less Multihop Network. In Proceedings of 3rd ACM International Symposiumon Mobile Ad Hoc Networking and Computing (MobiHoc’02), 2002.
[5] J. Bicket. Bit-rate Selection in Wireless Networks. Master’s thesis, MIT,Feb 2005.
[6] J. Bicket, D. Aguayo, S. Biswas, and R. Morris. Architecture and Evalu-ation of an Unplanned 802.11b Mesh Network. In Proceedings of 11th An-nual International Conference on Mobile Computing and Networking (Mo-biCom’05), 2005.
[7] V. Bychkovsky, B. Hull, A. Miu, H. Balakrishnan, and S. Madden. A Mea-surement Study of Vehicular Internet Access Using In Situ Wi-Fi Networks.In Proceedings of 12th Annual International Conference on Mobile Comput-ing and Networking (MobiCom’06), 2006.
[8] D. R. Choffnes and F. E. Bustamante. An integrated mobility and trafficmodel for vehicular wireless networks. In Proceedings of 2nd ACM Interna-tional Workshop on Vehicular Ad Hoc Networks (VANET’05), Sep 2005.
[9] S. Dashtinezhad and T. Nadeem. TrafficView: A Driver Assistant Devicefor Traffic Monitoring Based on Car-to-Car Communication. In IEEE Semi-annual Vehicular Technology, pages 17–19, 2004.
[10] O. Dousse, P. Thiran, and M. Hasler. Connectivity in Ad-hoc and HybridNetworks. In Proceedings of 21st IEEE Conference on Computer Commu-nications (INFOCOM’02), 2002.
137
[11] T. ElBatt, S. K. Goel, G. Holland, H. Krishnan, and J. Parikh. CooperativeCollision Warning Using Dedicated Short Range Wireless Communications.In Proceedings of 3rd ACM International Workshop on Vehicular Ad HocNetworks (VANET’06), 2006.
[12] R. Gass, J. Scott, and C. Diot. Measurements of In-Motion 802.11 Network-ing. In Proceedings of 8th IEEE Workshop on Mobile Computing Systemsand Applications (WMCSA’06), 2006.
[13] GLOMOSIM. http://pcl.cs.ucla.edu/projects/glomosim.
[14] C. Gorgorin, V. Gradinescu, R. Diaconescu, V. Cristea, and L. Iftode. AnIntegrated Vehicular and Network Simulator for Vehicular Ad-Hoc Networks.In Proceedings of 20th European Simulation and Modeling Conference, Oct2006.
[15] I. Haratcherev, J. Taal, K. Langendoen, R. Lagendijk, and H. Sips. Au-tomatic IEEE 802.11 Rate Control for Streaming Applications. WirelessCommunications and Mobile Computing Journal, June 2005.
[16] J. Harri, F. Filali, C. Bonnet, and M. Fiore. VanetMobiSim: GeneratingRealistic Mobility Patterns for VANETs. In Proceedings of 3rd ACM Inter-national Workshop on Vehicular Ad Hoc Networks (VANET’06) (Poster),Sep 2006.
[17] T. Henderson, D. Kotz, and I. Abyzov. The Changing Usage of a Ma-ture Campus-Wide Wireless Network. In Proceedings of 10th Annual Inter-national Conference on Mobile Computing and Networking (MobiCom’04),2004.
[18] G. Holland, N. Vaidya, and P. Bahl. A Rate-Adaptive MAC Protocol forMulti-Hop Wireless Networks. In Proceedings of 7th Annual InternationalConference on Mobile Computing and Networking (MobiCom’01), 2001.
[19] IEEE 802.11 Task Group n. Channel Models, May 2004.
[20] IEEE Std. 802.11. Wireless LAN Medium Access Control (MAC) and Phys-ical Layer (PHY) Specifications, 1999.
[21] S. Jaap, M. Bechler, and L. Wolf. Evaluation of Routing Protocols for Ve-hicular Ad Hoc Networks in City Traffic Scenarios. In Proceedings of 5thInternational Conference on Intelligent Transportation Systems Telecommu-nications (ITST’05), June 2005.
138
[22] G. Judd and P. Steenkiste. Using Emulation to Understand and ImproveWireless Networks and Applications. In Proceedings of 2nd Symposium onNetworked Systems Design & Implementation (NSDI’05), 2005.
[23] A. Kamerman and L. Monteban. WaveLAN-II: A High-Performance Wire-less LAN for the Unlicensed Band. Bell Labs Technical Journal, 1997.
[24] F. K. Karnadi, Z. H. Mo, and K. chan Lan. Rapid Generation of RealisticMobility Models for VANET. In Proceedings of IEEE Wireless Communi-cations and Networking Conference (WCNC’07), March 2007.
[25] B. Karp and H. T. Kung. GPSR: Greedy Perimeter Stateless Routing forWireless Networks. In Proceedings of 8th Annual International Conferenceon Mobile Computing and Networking (MobiCom’02), 2002.
[26] M. Killat, F. Schmidt-Eisenlohr, H. Hartenstein, C. Rossel, P. Vortisch,S. Assenmacher, and F. Busch. Enabling Efficient and Accurate Large-Scale Simulations of VANETs for Vehicular Traffic Management. In Pro-ceedings of 4th ACM International Workshop on Vehicular Ad Hoc Networks(VANET’07), Sep 2007.
[27] C. Krasic, J. Walpole, and W. Feng. Quality-Adaptive Media Streaming byPriority Drop. In Proceedings of 13th International Workshop on Networkand Operating Systems Support for Digital Audio and Video (NOSSDAV’03),2003.
[28] M. Lacage, M. H. Manshaei, and T. Turletti. IEEE 802.11 Rate Adapta-tion: A Practical Approach. In Proceedings of 7th International Sympo-sium on Modeling Analysis and Simulation of Wireless and Mobile Systems(MSWiM’04), 2004.
[29] P. Lambert, W. D. Neve, P. D. Neve, I. Moerman, P. Demeester, and R. V. D.Walle. Rate-Distortion Performance of H.264/AVC Compared to State-of-the-Art Video Codecs. IEEE Transactions on Circuits and Systems forVideo Technology, 6(1), Jan 2006.
[30] S. Lee, S. Banerjee, and B. Bhattacharjee. The Case for a Multi-hop WirelessLocal Area Network. In Proceedings of 23rd IEEE Conference on ComputerCommunications (INFOCOM’04), 2004.
[31] C. Lochert, B. Scheuermann, A. Barthels, A. Cervantes, M. Mauve, andM. Caliskan. Multiple Simulator Interlinking Environment for IVC. In Pro-ceedings of 2nd ACM International Workshop on Vehicular Ad Hoc Networks(VANET’05) (Poster), Sep. 2005.
139
[32] C. Lochert, B. Scheuermann, C. Wewetzer, A. Luebke, and M. Mauve. DataAggregation and Roadside Unit Placement for a VANET Traffic InformationSystem. In Proceedings of 5th ACM International Workshop on VehicularAd Hoc Networks (VANET’08), Sep 2008.
[33] J. Lubin. A Human Vision System Model for Objective Picture QualityMeasurement. In Sarnoff Corporation, USA.
[34] H. Luo, R. Ramjee, P. Sinha, L. Li, and S. Lu. UCAN: A Unified Cellular andAd-Hoc Network Architecture. In Proceedings of 9th Annual InternationalConference on Mobile Computing and Networking (MobiCom’03), 2003.
[35] Madwifi: Multiband Atheros Driver for WiFi.http://madwifi.sourceforge.net/.
[36] R. Mangharam, D. Weller, R. Rajkumar, P. Mudalige, and F. Bai.GrooveNet: A Hybrid Simulator for Vehicle-to-Vehicle Networks. In Pro-ceedings of 3rd Annual International Conference on Mobile and UbiquitousSystems, July 2006.
[37] R. Mangharam, D. S. Weller, D. Stancil, and R. Rajkumar. GrooveSim: ATopography-Accurate Simulator for Geographic Routing in Vehicular Net-works. In Proceedings of 2nd ACM International Workshop on Vehicular AdHoc Networks (VANET’05), 2005.
[38] S. McCanne and S. Floyd. Network simulator ns-2.
[39] MGEN. The Multi-Generator Toolset. http://mgen.pf.itd.nrl.navy.mil/.
[40] V. Naumov, R. Baumann, and T. Gross. An Evaluation of Inter-Vehicle AdHoc Networks Based on Realistic Vehicular Traces. In Proceedings of 7thACM International Symposium on Mobile Ad Hoc Networking and Comput-ing (MobiHoc’06), 2006.
[41] Netperf: A Network Performance Benchmark.http://www.netperf.org/netperf/netperfpage.html.
[42] OPNET. http://www.opnet.com.
[43] J. Ott and D. Kutscher. Drive-thru Internet: IEEE 802.11b for AutomobileUsers. In Proceedings of 23rd IEEE Conference on Computer Communica-tions (INFOCOM’04), 2004.
140
[44] K. Papagiannaki, M. Yarvis, and W. S. Conner. Experimental Characteri-zation of Home Wireless Networks and Design Implications. In Proceedingsof 25th IEEE Conference on Computer Communications (INFOCOM’06),2006.
[45] C. Perkins, E. Royer, and S. Das. Ad Hoc On Demand Distance Vec-tor (AODV) Routing. draft-ietf-manet-aodv-10.txt, Internet Draft, IETF,March 2002.
[46] M. Piorkowski, M. Raya, A. L. Lugo, P. Papadimitratos, M. Grossglauser,and J. P. Hubaux. TraNS: Realistic Joint Traffic and Network Simulatorfor VANETs. ACM SIGMOBILE Mobile Computing and CommunicationsReview, Issue 1, 12:31–33, Jan. 2008.
[47] PropSim. Propsim C8 - Wideband Multichannel Simulator.http://www.propsim.com/.
[48] QStream Project. Quality Adaptive Media Streaming. http://qstream.org.
[49] Qualnet. http://www.scalable-networks.com/.
[50] A. Qureshi and J. Guttag. Horde: Separating Network Striping Policy fromMechanism. In Proceedings of 3rd Annual International Conference on Mo-bile Systems, Applications and Services (MobiSys’05), 2005.
[51] G. F. Riley, R. M. Fujimoto, and M. H. Ammar. A Generic Frameworkfor Parallelization of Network Simulations. In Proceedings of 7th Interna-tional Symposium on Modeling, Analysis and Simulation of Computer andTelecommunication Systems (MASCOTS’99), 1999.
[52] P. M. Ruiz, F. J. Ros, and A. Gomez-Skarmeta. Internet Connectivity forMobile Ad Hoc Networks: Solutions and Challenges. IEEE Communica-tions, 43(10), 2005.
[53] B. Sadeghi, V. Kanodia, A. Sabharwal, and E. Knightly. OpportunisticMedia Access for Multirate Ad Hoc Networks. In Proceedings of 8th An-nual International Conference on Mobile Computing and Networking (Mo-biCom’02), 2002.
[54] A. K. Saha and D. B. Johnson. Modeling Mobility for Vehicular Ad-HocNetworks. In Proceedings of 1st ACM International Workshop on VehicularAd Hoc Networks (VANET’04) (Poster), Oct 2004.
141
[55] C. Schroth, F. Dotzer, T. Kosch, B. Ostermaier, and M. Strassberger. Sim-ulating the Traffic Effects of Vehicle-to-Vehicle Messaging Systems. In Pro-ceedings of 5th International Conference on ITS Telecommunications, June2005.
[56] S. Sen, O. Spatscheck, and D. Wang. Accurate, Scalable In-network Identi-fication of P2P Traffic Using Application Signatures. In Proceedings of 13thInternational World Wide Web (WWW) Conference, 2004.
[57] J. P. Singh, N. Bambos, B. Srinivasan, and D. Clawin. Cross-layer Multi-hop Wireless Routing for Inter-Vehicle Communication. In Proceedings ofIEEE/Create-Net Conference on Testbeds and Research Infrastructures forthe Development of Networks and Communities (TridentCom), 2006.
[58] M. Torrent-Moreno, P. Santi, and H. Hartenstein. Distributed Fair TransmitPower Adjustment for Vehicular Ad Hoc Networks. In Proceedings of 3rdAnnual IEEE Communications Society Conference on Sensor, Mesh and AdHoc Communications and Networks (SECON’06), Sep 2006.
[59] Urban Mobility Report 2005. Texas transportation institute.
[60] J. van der Merwe, S. Sen, and C. Kalmanek. Streaming Video Traffic: Char-acterization and Network Impact. In Proceedings of 7th International WebContent Caching and Distribution Workshop, 2002.
[61] VirtualDub. http://www.virtualdub.org/.
[62] VISSIM. http://www.ptv-vision.com/traffic/software-system-solutions/vissim/.
[63] VLC Media Player. http://www.videolan.org/vlc.
[64] S. Wang, C. Chou, Y. Chiu, Y. Tzeng, M. Hsu, Y. Cheng, W. Liu, and T. Ho.NCTUns 4.0: An Integrated Simulation Platform for Vehicular Traffic, Com-munication, and Network Researches. In Proceedings of 1st IEEE Interna-tional Symposium on Wireless Vehicular Communications (WiVEC’07), Sep2007.
[65] Wavelet. http://herbert.the-little-red-haired-girl.org/en/software/wavelet/.
[66] T. Wiegand, H. Schwarz, A. Joch, F. Kossentini, and G. J. Sullivan. Rate-Constrained Coder Control and Comparison of Video Coding Standards.In IEEE Transactions on Circuits and Systems for Video Technology, July2003.
142
[67] L. Wischhof, A. Ebner, and H. Rohling. Information Dissemination in Self-Organizing Intervehicle Networks. IEEE Transactions on Intelligent Trans-portation Systems, 6(1):90–101, March 2005.
[68] H. Wu, J. Lee, M. Hunter, R. Fujimoto, R. L. Guensler, and J. Ko. Efficiencyof Simulated Vehicle-toVehicle Messge Propagation in Atlanta, Georgia I-75Corridor. Transportation Research Record: Journal of The TransportationResearch Board, (1910):82–89, 2005.
[69] H. Wu, M. Palekar, R. Fujimoto, R. Guensler, M. Hunter, J. Lee, and J. Ko.An Empirical Study of Short Range Communications for Vehicles. In Pro-ceedings of 2nd ACM International Workshop on Vehicular Ad Hoc Networks(VANET’05), 2005.
[70] Z. Wu, S. Ganu, I. Seskar, and D. Raychaudhuri. Experimental Investigationof PHY Layer Rate Control and Frequency Selection in 802.11-based Ad-Hoc Networks. In Proceedings of Sigcomm 2005 Workshop on ExperimentalApproaches to Wireless Network Design and Analysis (E-WIND’05), 2005.
[71] J. Yoon, M. Liu, and B. Noble. Random Waypoint Considered Harmful. InProceedings of 22nd IEEE Conference on Computer Communications (IN-FOCOM’03), 2003.
[72] J. Zhou, Z. Ji, and R. Bagrodia. TWINE: A Hybrid Emulation Testbed forWireless Networks and Applications. In Proceedings of 25th IEEE Confer-ence on Computer Communications (INFOCOM’06), April 2006.
143
