Experimental and Simulation Investigation on the TCP Performance for Wireless Broadband

EnvironmentA. Rufini, M. Giuntini, A. Valenti

Fondazione Ugo Bordoni Viale del Policlinico 147, 00161 Rome, Italy

Donato Del Buono

ISCOM Viale America 201, 00144 Rome, Italy

Abstract—Next Generation Networks provide opportunities and challenges for data communication. In particular, the ever increasing bandwidth poses a major challenge to transport layer protocols such as TCP. In this paper the performance of TCP in large bandwidth delay networks, with particular regard to wireless network scenarios, is experimentally investigated. In particular we apply an accredited standard technique for end-user bandwidth evaluation in wireless access environments showing limitations in the bandwidth exploitation. This investigation shows how end-to-end bandwidth measurements depend strongly on the TCP implementation in a broadband access network. We also simulated the experimental environment by means of ns-3 code, reproducing the same experimental results. This simulation approach allows us to foresee scenarios that cannot be analyzed in laboratories and to investigate future 5G infrastructures.

Keywords—Network performance evaluation; wireless broadband access networks, TCP; ns-3; QoS

I. INTRODUCTION The development of next generation services will be more

and more based on the implementation of ultra broadband accesses, with a wide component of wireless devices for the convergence towards 5G infrastructures [1]. Most of the services are based on Transmission Control Protocol (TCP), a very reliable transport layer [2] and the most adopted protocol in the Internet that in high bandwidth-delay product networks suffers several limitations in the bandwidth exploitation. In wireless environments, where loss, delay and jitter have strong fluctuations in time and space TCP performance could be further degraded. As well known, the protocol supports reliable data transfer by establishing a connection between the transmitting and receiving ends. Although TCP was initially designed and optimized for wired networks, currently it is deeply used also for wireless broadband, due to the growing popularity of wireless data applications and the growth of wireless devices and variety of services provided. With respect to a wireline link, a wireless one is generally characterized by high transmission errors, random interference and high varying latency due to channel fading and user mobility. When TCP is employed for data transport in such environments, highly RTTs (Round Trip Time) can induce spurious timeouts, even though the involved packet actually is not lost but simply delayed [3]. RTT is the time it takes for a signal to be sent plus the time it takes for an acknowledgement of that signal to be received. In addition relevant network RTT can delay consistently the

achievement of the connection steady state, making it unreachable for short-lived flows. In this regard, it is important to investigate intrinsic limitations of TCP performance related to inner protocol algorithms, in order to detect the key parameters and mechanisms to further improve TCP performance in radio mobile networks. In order to carry out a performance analysis in a common reference framework we refer to a bandwidth estimation method proposed by European Telecommunications Institute (ETSI) in [4] and we report an experimental investigation about the impact of specific parameters of the wireless environment on the performance of TCP protocol.

In this work, we are interested only in the achievable end-to-end TCP throughput so an experimental evaluation is carried out analyzing performance and characteristics of a single TCP connection through experimental conditions typical of radio access. In addition we propose a complete comparison between real test-bed measurements and probes emulated by means of the simulation platform Network Simulator ns-3 [5]. Results confirm reliability of ns-3 for further investigations, and therefore the reliability of such a tool to investigate 5G networks. The paper is organized as follows. Section II presents a brief description of the methods to measure the user Quality of Service (QoS) in wireless Broadband Access Networks. Section III provides details about the adopted experimental and simulation setup for different application scenarios. In section IV an extensive comparison, based on simulation results obtained from the test bed setup and the ones by ns-3 network simulator, is reported. Finally, conclusions are drawn in section V.

II. TCP PERFORMANCE IN WIRELESS NETWORKS Today bandwidth estimation plays a key role in

telecommunication evolution and market regulation in order to correctly define service level agreements between customers and Internet Service Providers (ISP). On the other hand customers are interested in network performance verification to better exploit ISP market competition. For these reasons this study aims for investigating bandwidth estimation techniques based on active probes designed considering protocols commonly used by customers: in this way bandwidth estimation matches the experienced performance closer to the real user experience. In this field some studies are already carried out in [6, 7], regarding measurements on ADSL2+ that is the most adopted technology by

Telecommunication Operators in Europe. Furthermore, in [8] we report an experimental investigation on the bandwidth measurements and capacity exploitation in Gigabit Passive Optical Networks (GPON) and the limitations caused by the Internet protocols and we also investigated on use of UDP (User Datagram Protocol) and multisession TCP. In [9] we also report an experimental investigation on QoS measurements including a correlation in terms of QoE (Quality of Experience). Finally, some studies regarding the monitoring of the QoS performance provided by Italian mobile network operators have been carried out, in particular an analysis based on drive-test measurements for the evaluation of the impact of mobility on typical KPIs (Key Performance Indicators) used for the assessment of QoS in mobile networks [10].

Even though this study is related to wireless environment, to carry out our experimental investigation a fixed access network, such as ADSL2+, has been adopted. This choice rises from the need to directly control parameters and consequently network performances, excluding all random phenomena typical of wireless environment. To better carry out this investigation, typical parameters of the radio access, such as high RTT, have been taken into account.

Wireless environments pose formidable challenges when attempting to provide reliable, end-to-end data transmission for transport protocols such as TCP. In particular wireless networks have peculiar characteristics that can have a negative impact on the performance of transmission such as random packet loss, low channel capacity, high RTT. In the assumption above mentioned, the TCP plays a key role in evaluating network performance, since it directly regulates the data flow. In particular, the TCP receiver sends back an acknowledgement for every received data segment, ensuring the proper execution of communication. On the other end of the connection, if the transmitter does not receive an acknowledgement for a given packet when the corresponding timeout period expires, the packet is considered lost and subject to retransmission. To take into account network conditions, the transmitter starts a timeout mechanism when sending a packet to the receiver and constantly tracks RTTs for its packets as a means to determine the appropriate timeout period. Moreover TCP implements some proactive mechanisms to prevent packet loss trying to avoid exceeding the network and receiver capacity: Congestion Control (rfc 2581) [11] and Flow Control (rfc 793) [12], which cooperate data transfer tuning.

The choice of a TCP dependent technique for bandwidth estimation tries to keep QoS evaluation as closer as possible to the end-user effective experience of broadband access services. In particular the maximum throughput of a TCP flow directly depends on the congestion window size; if the congestion window is too large, there is a high probability of packet loss because the network and the receiver have resource limitations. As known, at the start of a new TCP connection, the sender does not know the most suitable congestion window for the path, so in the slow start phase the congestion window grows exponentially, doubling every Round Trip Time until reach the threshold value. This value is an arbitrary default value depending on the operating system implementation; this setting

is critical because it influences the TCP performance. In fact if the threshold value is set too high relatively to the network Bandwidth Delay Product, the exponential increase of congestion window generates many packets, causing multiple losses with significant reduction of the connection throughput. On the other hand if it is too low, TCP may need a very long time to reach the proper window size resulting in poor start-up utilization especially when Bandwidth Delay Product is large as in wireless communications networks. Thus, minimization of packet losses requires minimizing the congestion window. Therefore, the problem is finding an optimal value for the congestion window (which is usually referred to as the congestion window) that provides good throughput, yet does not overwhelm the network and the receiver capacity. Additionally, TCP should be able to recover from packet losses in a timely fashion. This means that the shorter the interval between packet transmission and loss detection, the faster TCP can recover. However, this interval cannot be too short, since the sender could detect a loss prematurely and retransmit the corresponding packet unnecessarily.

The estimation of the average TCP throughput, Rate, can be assumed as

Rate = CWIN / RTT [byte/s] (1)

Where CWIN = window size * segment size [byte] and RTT=(packet transmission time + link propagation delay)*2 [s]. This estimation requires the knowledge of the following parameters: window size (segment), segment size (byte), link propagation delay (ms), packet transmission time (ms).

To maximize the link transmission capacity it is necessary to transmit a quantity of information equal to Bandwidth - Delay product (BDP), expressed by the following relationship:

BDP (bytes) = bandwidth (kbyte/s) * RTT (ms) (2)

To get full TCP performance, the ideal TCP window size needs to be large enough to accommodate the BDP and it represents the amount of information that uses completely the link between the transmitter and receiver. Therefore, the optimal receive window size (Wid) for TCP flow control is equal to the maximum number of bits that the connection can hold at the same RTT divided by the size of each segment:

Wid = (bandwidth * RTT) / segment size (3)

III. EXPERIMENTAL AND SIMULATION SETUP

A. Test Bed In order to control the experimental condition we adopted a

wireline test bed setting typical wireless parameters. The network test bed is composed by ADSL2+ systems for the access part of the network; the ADSL2+ system consists of an Alcatel DSLAM (ISAM 7324) that allows us to use downstream links with different bit rates. According to our tests, in Figure 1 a detail of the reference topology is illustrated; it represents a segment of Metropolitan Area Network with a core and an access part: a client accesses the network through ADSL2+ technology and communicates with a server that is linked to router Cisco 3. Furthermore there is a delay generator (Shunra Wan Emulator) between server and client that provides different and high RTTs, reproducing one of the key characteristic of the mobile environment.

Fig. 1. Test Bed Architecture.

Fig. 2. ns-3 Network Topology.

The aim is to outline the impact of TCP implementation in QoS evaluation and bandwidth customer exploitation.

B. Network simulator ns-3 To confirm the obtained results with the experimental

setup, we simulated the same TCP data transfer with ns-3 network simulator. Ns-3 is a discrete event network simulator useful for the study of Internet protocols and large scale systems. In particular we have chosen ns-3 simulator because it allows us to implement different TCP configurations. Furthermore, ns-3 supports the Network Simulation Cradle (NSC) [13] for wrapping real-world network code into simulators, allowing simulation of real-world behaviour. Moreover the results, obtained with this simulator, require much lower simulation time compared with the same ones obtained by test bed. The ns-3 simulator efficiency can be measured in terms of simulation time reduction and possibility of implementing new network scenarios with devices not available in lab. The adopted network topology is shown in Figure 2, N0 and N1 represent respectively a sender and a TCP receiver. The channel between the two nodes is a P2P channel. The results, reported in the successive section, are obtained for varying propagation time and channel capacity.

IV. RESULTS The following two paragraphs show results in terms of

network performance obtained by experimental and simulation tests. To conclude a comparison between two results has been proposed.

A. Experimental results The current paragraph shows results obtained from a set of

experimental tests. This section analyzes the current TCP Slow start mechanisms and evaluates their startup performance in high delay networks.

All results are obtained using iperf [14], a network testing tool which is used to measure the end-to-end achievable bandwidth using TCP data streams and allowing variations in parameters like TCP window size and number of parallel streams. In particular it has a client and server functionality, and can determine the throughput between the two ends, either unidirectional or bi-directionally. Iperf allows the user to set different parameters that can be used for testing a network, either alternately for optimizing or tuning a network.

Fig. 3. BDP impact over Measurement Efficiency

Results collected in this experimental analysis are referred to different access network scenarios in terms of available bit rate, delay and test duration that characterize access technology and protocols performance. Throughput tests are carried out establishing an end-to-end TCP connection considering different test duration and introducing different network delays.

In Figure 3 the obtained results in terms of Measurement Ratio are reported, representing an efficiency index that quantify the difference between estimated throughput and layer 2 maximum bandwidth (Line Capacity) provided by access network. Figure details the trend of Measurement Ratio, in correspondence of different increasing BDP values (2). Each curve represents a test of different duration and TCP window size.

This analysis wants to highlight the role of connection startup (TCP slow-start) on the overall connection throughput; the goal is achieved considering different long-lived connections and their performance.

It is possible to see how the growth of test duration (e.g. 30sec) increases the measured throughput percentage values especially in presence of high delay. This effect is due to the TCP startup algorithm that grows progressively the data rate in order to avoid network overloading [15].

Figure 4 show the trend of the Congestion window in the startup stage and the relative throughput to this configuration. In particular the receiver issues an Acknowledgement (ACK) for every data packet received. We assume the receiver’s advertised window is always large so that the actual sending window is always equal to CWIN. For convenience, the window size is measured in number of packets, and the packet size is 1460 bytes. In TCP mechanism, a sender starts in slow-start, CWIN < SSTHRESH and every ACK received results in an increase of CWIN by 1 packet. Thus, the sender exponentially increases CWIN. When CWIN hits the Slow Start Threshold value (SSTHRESH), the sender switches to congestion avoidance phase, increasing CWIN linearly, considerably slower than in slow start. We need to run iperf for fairly long periods of time to counter the effects of TCP slow-start. By definition the duration of slow-start is roughly [log2 (ideal_window_size_in_MSS)] * RTT. In our scenario, the bandwidth-delay product for 7 Mbps network with an RTT of 150 ms is about 90 segments, 1460-byte.

Fig. 4. Throughput and Congestion Window.

The slow-start duration will be approximately [log2 (90) *0.15], which is 1 second. Assuming a stable congestion window after slow-start, the time for the cumulative bandwidth to reach 90% of the achievable bandwidth will be about [10*slow_start_duration–10*RTT]. This means for the 7Mbps-150ms network path, it will take over 8.5 seconds for the cumulative bandwidth to reach 90% of the achievable bandwidth.

To confirm this model, a detailed passive monitoring was carried out by using tcptrace, a tool [16] written by Shawn Ostermann at Ohio University. It analyzes TCP dump files taking as input the files produced by several popular packet-capture programs. Thanks to this tool it is possible to produce several different types of output containing information on each connection seen, such as elapsed time, bytes and segments sent and received, retransmissions, round trip times, window advertisements, throughput, and more. Results represented in figure 4 report on the Y-axis the throughput and the in-flight data respectively and the X-axis the connection time. At the top of the figures, the upper red line tracks the throughput seen from the last few samples, calculated as the average of N previous dots (these points are not shown for intelligibility of the figure). The green line represents the average throughput of the connection up to that point in the life time of the connection (total bytes seen/total seconds so far).

The idea behind the graphs at the bottom of figures 4a and 4b is to estimate the congestion window (CWIN) at the sender. Since this cannot be determined accurately, we use the in-flight unacknowledged data as estimation.

•Green line tracks the window advertised by the opposite end-point i.e., the receive window (RWIN);

•Blue line represents instantaneous in-flight data samples at various points in the lifetime of the connection (CWIN);

•Yellow line tracks the average in-flight data up to that point;

•Red line tracks the weighted average of in-flight data up to that point.

These results are carried out for a bottleneck bandwidth of 7Mbps, for RTT values of 150ms and for different test duration. The buffer size is set equal to BDP in each case. Figure 4a shows that a 3second measurement is not sufficient for a 7Mbps-150ms bandwidth-delay network path, since the slow start duration is comparable with the session duration. As a consequence, the achieved throughput is only 4.5 Mbps, much lower than the actual 7 Mbps. In Figure 4b, instead, when measurement test increases (30sec), the achieved throughput for the same operative scenario is equal to 5.9 Mbps.

Considering a 7Mbps-750ms network path (Fig. 5), the results for a 3second measurement have not been reported due to the heavy network delay impact on the slow start algorithm. In particular the slow-start duration is about 13sec. For a 30second measurement, we can observe that when RTT=750ms, TCP stops exponentially growing CWIN long before it reaches the ideal value (BDP=656Kbyte). After that, CWIN increases slowly, and has not reach 450 packets by 30sec. As a result, the achieved throughput is only 3.8 Mbps, much lower than the desired 7 Mbps. To conclude like other window-based protocols, TCP performances depend on RTT value. In particular when RTT increases (e.g. 750ms), the ideal window grows too. On the other hand, because CWIN increases 1 packet per RTT during congestion-avoidance, longer RTT means slower CWIN growth, resulting in even lower utilization.

Fig. 5. 7Mb 750 RTT duration test 30sec.

B. Simulation results By using the ns-3 simulator in figures 6 and 7, the most

important results are shown. In particular these results have been obtained by implementing a TCP application in ns-3 that generates a CBR traffic flow from N0 to N1. Then we worked out a pcap trace with tcptrace to analyze the TCP performance in terms of throughput and congestion window. As well known from TCP theory, TCP connections can take several cycles to reach a steady state, so the short connections ("short-lived connections") are not well modeled, as it is possible to see from figure above reported (Fig. 4a). In this regard figures 6 and 7 show only test results of 30sec duration.

Fig. 6. 7Mb 150 RTT duration test 30sec.

Fig. 7. 7Mb 750 RTT duration test 30sec.

We can see how the throughput and congestion window trend obtained through ns-3 simulator are quite similar with the same ones achieved on the test bed. In particular the average throughput is around 5.9 Mbps and 3.8 Mbps when the RTT is equal to 150ms and 750ms respectively.

Fig. 8. Test Bed vs ns-3: Throughput and CWIN.

To confirm these results, in the above figure (Fig. 8), a comparison between the throughput and congestion window curves obtained by the test bed (red lines) and the ones by ns-3 network simulator (blue lines) has been reported. Considering both laboratory and simulation results, we pointed out the relevance of the measure duration in addition to the importance of optimal window size.

V. CONCLUSION In this work we have analyzed the TCP protocol

performance as a function of throughput parameters. As known from theory, the overall throughput of window protocols depends on the connection duration. The obtained results show how the TCP session duration is an important parameter in order to fully exploit the available bandwidth. This consideration heavily impacts in performance measurements based on TCP, as a consequence an accurate tuning of the session duration is mandatory to perform an effective bandwidth evaluation. In particular, session duration dimensioning must to take into consideration the slow start period of the TCP session. This is a focal point especially considered the high capacity that can reach the new mobile 4G and future 5G technologies. Considering both laboratory and simulation results, it is been pointed out the relevance of the measure duration in addition to the importance of optimal window size. The choice of the test duration and connection parameters can be implemented in a simulation environment such as ns-3, in order to perform bandwidth test on TCP connection in wireless environments. The obtained results highlight how this simulator is representative of the dynamics that characterize the TCP performance in wireless environments. This is particularly interest because results obtained with simulation require much lower simulation time

compared with the same ones obtained by test bed. The comparison between experimental and simulation results has shown similar results, so the simulation tool can be assumed as reliable and useful to investigate network scenarios with devices not available in lab. To conclude, the ns-3 simulator efficiency can be measured in terms of simulation times reduction and possibility of implementing new network scenarios that allow to face the ever increasing evolution of mobile broadband networks and services.

ACKNOWLEDGMENT

This work was carried out within the framework of ATENA project.

REFERENCES [1] Hossain, E., Hasan, M., “5G Cellular:Key Enabling Technologies and

Research Challenges,” IEEE Instrumentation & Measurement Magazine, 2015 (June 2015).

[2] Afanasyev, A., et al., “Host-to-Host Congestion Control for TCP,” IEEE Communications Surveys & Tutorials, Vol. 12, NO.3, Third Quarter 2010.

[3] Leung, K., et al., “Methods to Improve TCP Throughput in Wireless Networks With High Delay Variability,” in Vehicular Technology Conference, 2004. VTC2004-Fall. 2004 IEEE 60th.

[4] ETSI EG 202 057-4 User related QoS parameter definitions and measurement. European Telecommunications Standards Institute (October 2005), http://www.etsi.org.

[5] NS-3 network simulator (ver. ns-3.10) URL: http://www.nsnam.org. [6] Bolletta, P., et al., “On the Impact of Operative Systems Choice in End-

user Bandwidth Evaluation: Testing and Analysis in a Metro-access Network,” IARIA ACCESS 2010, Valencia, Spain (September 2010).

[7] Bolletta, P., et al., “Monitoring of the User Quality of Service: Network Architecture for Measurements and role of Operating System with consequences for optical accesses,” ONDM 2011, Bologna, Italy (February 2011).

[8] Rufini, A., Mellia, M., Tego, E., Matera, F., “Multilevel Bandwidth Measurements and Capacity Exploitation in Gigabit Passive Optical Networks,” IET Communications, 2014, Vol. 8, Iss. 18, pp. 3357–3365.

[9] Rufini, A., Tego, E., Matera, F., “Multilevel QoS vs QoE measurements and Verification of Service Level Agreements,” EuCNC2014, Bologna, Italy (June 2014).

[10] Rufini, A., et al., “Evaluation of the impact of mobility on typical KPIs used for the assessment of QoS in mobile networks: an analysis based on drive-test measurements,” Netowrks2014, Madeira, Portugal (September 2014).

[11] M. Allman, V. Paxson, and W. Stevens, “RFC2581—TCP congestion control,” RFC, 1999.

[12] J. Postel. RFC 793: Transmission control protocol, September 1981. Status: STANDARD.

[13] Network Simulation Cradle URL: http://www.wand.net.nz/~stj2/nsc. [14] Iperf (ver. 2.0.4) URL: http://sourceforge.net/projects/iperf. [15] W. Stevens. RFC 2001: TCP slow start, congestion avoidance, fast

retransmit, and fast recovery algorithms, January 1997. Status: PROPOSED STANDARD.

[16] http://www.tcptrace.org/manual.html.