Post on 18-Mar-2018
transcript
Master ThesisElectrical EngineeringOctober 2013
QoE of Video Streaming over LTE Network
Pradeep Uppu Sushanth Kadimpati
School of ComputingBlekinge Institute of Technology37179 KarlskronaSweden
This thesis is submitted to the School of Computing at Blekinge Instituteof Technology in partial fulfilment of the requirements for the degree ofMaster of Science in Electrical Engineering The thesis is equivalent to 20weeks of full time studies
This Master Thesis is typeset using LATEX
Contact Information
Author(s)Pradeep UppuAddress Karlskrona SwedenE-mail pradeepuppugmailcomSushanth KadimpatiAddress Karlskrona SwedenE-mail sushanthkadgmailcom
University advisor(s)
Dr Adrian Popescu ProfCOMBTH
School of ComputingBlekinge Institute of Technology371 79 KARLSKRONA SWEDEN
Internet wwwbthsecomPhone +46 455 385000SWEDEN
Abstract
In recent years the mobile Internet has increased dramatically with thedevelopment of 3G and 4G technologies Especially the usage of mobilebroadband internet on the devices like cellular mobiles Tablets and Laptopshas skyrocketed Among the multimedia applications video streaming is themost popular mobile application But making these services available tousers in a cost effective way without compromising quality is a big challengeThe development of Long Term Evolution (LTE) technology in the mobileworld made this task achievable The features of LTE technology provideeffective services in multimedia applications with high data rates and lowlatency
In this paper we study and analyze the Quality of Experience (QoE)at the end user for Video on Demand (VoD) over the LTE network Toachieve this we streamed High Definition (HD) videos based on H264AVCand these videos are delivered from source to destination using TransportControl Protocol (TCP) and User Datagram Protocol (UDP) Specificallyour study is about QoE evaluation in terms of delay variation packet lossmetrics and provides performance evaluation to characterize the impact oftransport layer protocol in video streaming over radio networks like LTEIn order to know the performance of video streaming over LTE network wealso evaluate the LTE performance in terms of one-way delay packet lossand inter packet delay for the generated UDP and TCP packets
Keywords QoE Video Streaming H264AVC LTE One-way DelayPacket loss
i
Acknowledgements
It gives us great immense joy in acknowledging Prof Adrian Popescu forhis diligent support and extending the opportunity to pursue our masterthesis under his immaculate supervision We would like to thank Dr PatrikArlos for providing the experimental test bed and valuable suggestions in theevolution of this thesis We would also like to thank Wowza Media Systemsfor providing the software A word of thanks to Mr Tahir Minhas Nawazfor his valuable suggestions and tips We are indebted to our friends andfamily for their constant support and prayers that helped us complete thethesis
This thesis is dedicated to our parents who stood beside us through thickand thin in making this thesis substantial
Pradeep UppuSushanth Kadimpati
ii
Contents
Abstract i
Acknowledgements ii
Contents ii
List of Figures vi
List of Tables viii
Acronyms ix
1 Introduction 211 Motivation 312 Related Works 413 Contribution 614 Aims and Objectives 615 Research Questions 716 Thesis Outline 7
2 Technical Background 921 Quality of Experience 922 Video Streaming 923 Video Compression 1024 Video Format 11
241 Video Codec 11242 Video Container 11243 H264AVC Codec 11
25 Types of Video Streaming 1226 Supported Protocols for Video Streaming 1227 Assessment of Videos 12
iii
28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16
2101 Core Network 162102 Radio Access Network 17
3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20
311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22
32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23
33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27
4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30
411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31
412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34
413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35
4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36
415 Inter Packet Delay for TCP packets in Uplink 37
iv
4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38
416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41
451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45
452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49
5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52
Bibliography 53
v
List of Figures
21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18
31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27
41 Minimum One way Delay for Generated UDP Packets forUplink 30
42 Maximum One way Delay for Generated UDP Packets forUplink 31
43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for
Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for
Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for
Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for
Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for
Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-
ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet
size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43
vi
418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49
vii
List of Tables
21 Five-level scale for rating overall quality of video 13
31 Test Video Parameters 26
41 MOS for Delay Variation 4342 MOS for Packet Loss 46
viii
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
This thesis is submitted to the School of Computing at Blekinge Instituteof Technology in partial fulfilment of the requirements for the degree ofMaster of Science in Electrical Engineering The thesis is equivalent to 20weeks of full time studies
This Master Thesis is typeset using LATEX
Contact Information
Author(s)Pradeep UppuAddress Karlskrona SwedenE-mail pradeepuppugmailcomSushanth KadimpatiAddress Karlskrona SwedenE-mail sushanthkadgmailcom
University advisor(s)
Dr Adrian Popescu ProfCOMBTH
School of ComputingBlekinge Institute of Technology371 79 KARLSKRONA SWEDEN
Internet wwwbthsecomPhone +46 455 385000SWEDEN
Abstract
In recent years the mobile Internet has increased dramatically with thedevelopment of 3G and 4G technologies Especially the usage of mobilebroadband internet on the devices like cellular mobiles Tablets and Laptopshas skyrocketed Among the multimedia applications video streaming is themost popular mobile application But making these services available tousers in a cost effective way without compromising quality is a big challengeThe development of Long Term Evolution (LTE) technology in the mobileworld made this task achievable The features of LTE technology provideeffective services in multimedia applications with high data rates and lowlatency
In this paper we study and analyze the Quality of Experience (QoE)at the end user for Video on Demand (VoD) over the LTE network Toachieve this we streamed High Definition (HD) videos based on H264AVCand these videos are delivered from source to destination using TransportControl Protocol (TCP) and User Datagram Protocol (UDP) Specificallyour study is about QoE evaluation in terms of delay variation packet lossmetrics and provides performance evaluation to characterize the impact oftransport layer protocol in video streaming over radio networks like LTEIn order to know the performance of video streaming over LTE network wealso evaluate the LTE performance in terms of one-way delay packet lossand inter packet delay for the generated UDP and TCP packets
Keywords QoE Video Streaming H264AVC LTE One-way DelayPacket loss
i
Acknowledgements
It gives us great immense joy in acknowledging Prof Adrian Popescu forhis diligent support and extending the opportunity to pursue our masterthesis under his immaculate supervision We would like to thank Dr PatrikArlos for providing the experimental test bed and valuable suggestions in theevolution of this thesis We would also like to thank Wowza Media Systemsfor providing the software A word of thanks to Mr Tahir Minhas Nawazfor his valuable suggestions and tips We are indebted to our friends andfamily for their constant support and prayers that helped us complete thethesis
This thesis is dedicated to our parents who stood beside us through thickand thin in making this thesis substantial
Pradeep UppuSushanth Kadimpati
ii
Contents
Abstract i
Acknowledgements ii
Contents ii
List of Figures vi
List of Tables viii
Acronyms ix
1 Introduction 211 Motivation 312 Related Works 413 Contribution 614 Aims and Objectives 615 Research Questions 716 Thesis Outline 7
2 Technical Background 921 Quality of Experience 922 Video Streaming 923 Video Compression 1024 Video Format 11
241 Video Codec 11242 Video Container 11243 H264AVC Codec 11
25 Types of Video Streaming 1226 Supported Protocols for Video Streaming 1227 Assessment of Videos 12
iii
28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16
2101 Core Network 162102 Radio Access Network 17
3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20
311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22
32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23
33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27
4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30
411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31
412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34
413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35
4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36
415 Inter Packet Delay for TCP packets in Uplink 37
iv
4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38
416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41
451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45
452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49
5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52
Bibliography 53
v
List of Figures
21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18
31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27
41 Minimum One way Delay for Generated UDP Packets forUplink 30
42 Maximum One way Delay for Generated UDP Packets forUplink 31
43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for
Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for
Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for
Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for
Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for
Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-
ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet
size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43
vi
418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49
vii
List of Tables
21 Five-level scale for rating overall quality of video 13
31 Test Video Parameters 26
41 MOS for Delay Variation 4342 MOS for Packet Loss 46
viii
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Abstract
In recent years the mobile Internet has increased dramatically with thedevelopment of 3G and 4G technologies Especially the usage of mobilebroadband internet on the devices like cellular mobiles Tablets and Laptopshas skyrocketed Among the multimedia applications video streaming is themost popular mobile application But making these services available tousers in a cost effective way without compromising quality is a big challengeThe development of Long Term Evolution (LTE) technology in the mobileworld made this task achievable The features of LTE technology provideeffective services in multimedia applications with high data rates and lowlatency
In this paper we study and analyze the Quality of Experience (QoE)at the end user for Video on Demand (VoD) over the LTE network Toachieve this we streamed High Definition (HD) videos based on H264AVCand these videos are delivered from source to destination using TransportControl Protocol (TCP) and User Datagram Protocol (UDP) Specificallyour study is about QoE evaluation in terms of delay variation packet lossmetrics and provides performance evaluation to characterize the impact oftransport layer protocol in video streaming over radio networks like LTEIn order to know the performance of video streaming over LTE network wealso evaluate the LTE performance in terms of one-way delay packet lossand inter packet delay for the generated UDP and TCP packets
Keywords QoE Video Streaming H264AVC LTE One-way DelayPacket loss
i
Acknowledgements
It gives us great immense joy in acknowledging Prof Adrian Popescu forhis diligent support and extending the opportunity to pursue our masterthesis under his immaculate supervision We would like to thank Dr PatrikArlos for providing the experimental test bed and valuable suggestions in theevolution of this thesis We would also like to thank Wowza Media Systemsfor providing the software A word of thanks to Mr Tahir Minhas Nawazfor his valuable suggestions and tips We are indebted to our friends andfamily for their constant support and prayers that helped us complete thethesis
This thesis is dedicated to our parents who stood beside us through thickand thin in making this thesis substantial
Pradeep UppuSushanth Kadimpati
ii
Contents
Abstract i
Acknowledgements ii
Contents ii
List of Figures vi
List of Tables viii
Acronyms ix
1 Introduction 211 Motivation 312 Related Works 413 Contribution 614 Aims and Objectives 615 Research Questions 716 Thesis Outline 7
2 Technical Background 921 Quality of Experience 922 Video Streaming 923 Video Compression 1024 Video Format 11
241 Video Codec 11242 Video Container 11243 H264AVC Codec 11
25 Types of Video Streaming 1226 Supported Protocols for Video Streaming 1227 Assessment of Videos 12
iii
28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16
2101 Core Network 162102 Radio Access Network 17
3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20
311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22
32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23
33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27
4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30
411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31
412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34
413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35
4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36
415 Inter Packet Delay for TCP packets in Uplink 37
iv
4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38
416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41
451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45
452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49
5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52
Bibliography 53
v
List of Figures
21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18
31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27
41 Minimum One way Delay for Generated UDP Packets forUplink 30
42 Maximum One way Delay for Generated UDP Packets forUplink 31
43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for
Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for
Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for
Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for
Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for
Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-
ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet
size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43
vi
418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49
vii
List of Tables
21 Five-level scale for rating overall quality of video 13
31 Test Video Parameters 26
41 MOS for Delay Variation 4342 MOS for Packet Loss 46
viii
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Acknowledgements
It gives us great immense joy in acknowledging Prof Adrian Popescu forhis diligent support and extending the opportunity to pursue our masterthesis under his immaculate supervision We would like to thank Dr PatrikArlos for providing the experimental test bed and valuable suggestions in theevolution of this thesis We would also like to thank Wowza Media Systemsfor providing the software A word of thanks to Mr Tahir Minhas Nawazfor his valuable suggestions and tips We are indebted to our friends andfamily for their constant support and prayers that helped us complete thethesis
This thesis is dedicated to our parents who stood beside us through thickand thin in making this thesis substantial
Pradeep UppuSushanth Kadimpati
ii
Contents
Abstract i
Acknowledgements ii
Contents ii
List of Figures vi
List of Tables viii
Acronyms ix
1 Introduction 211 Motivation 312 Related Works 413 Contribution 614 Aims and Objectives 615 Research Questions 716 Thesis Outline 7
2 Technical Background 921 Quality of Experience 922 Video Streaming 923 Video Compression 1024 Video Format 11
241 Video Codec 11242 Video Container 11243 H264AVC Codec 11
25 Types of Video Streaming 1226 Supported Protocols for Video Streaming 1227 Assessment of Videos 12
iii
28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16
2101 Core Network 162102 Radio Access Network 17
3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20
311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22
32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23
33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27
4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30
411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31
412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34
413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35
4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36
415 Inter Packet Delay for TCP packets in Uplink 37
iv
4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38
416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41
451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45
452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49
5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52
Bibliography 53
v
List of Figures
21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18
31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27
41 Minimum One way Delay for Generated UDP Packets forUplink 30
42 Maximum One way Delay for Generated UDP Packets forUplink 31
43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for
Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for
Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for
Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for
Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for
Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-
ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet
size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43
vi
418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49
vii
List of Tables
21 Five-level scale for rating overall quality of video 13
31 Test Video Parameters 26
41 MOS for Delay Variation 4342 MOS for Packet Loss 46
viii
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Contents
Abstract i
Acknowledgements ii
Contents ii
List of Figures vi
List of Tables viii
Acronyms ix
1 Introduction 211 Motivation 312 Related Works 413 Contribution 614 Aims and Objectives 615 Research Questions 716 Thesis Outline 7
2 Technical Background 921 Quality of Experience 922 Video Streaming 923 Video Compression 1024 Video Format 11
241 Video Codec 11242 Video Container 11243 H264AVC Codec 11
25 Types of Video Streaming 1226 Supported Protocols for Video Streaming 1227 Assessment of Videos 12
iii
28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16
2101 Core Network 162102 Radio Access Network 17
3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20
311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22
32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23
33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27
4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30
411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31
412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34
413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35
4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36
415 Inter Packet Delay for TCP packets in Uplink 37
iv
4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38
416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41
451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45
452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49
5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52
Bibliography 53
v
List of Figures
21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18
31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27
41 Minimum One way Delay for Generated UDP Packets forUplink 30
42 Maximum One way Delay for Generated UDP Packets forUplink 31
43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for
Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for
Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for
Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for
Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for
Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-
ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet
size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43
vi
418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49
vii
List of Tables
21 Five-level scale for rating overall quality of video 13
31 Test Video Parameters 26
41 MOS for Delay Variation 4342 MOS for Packet Loss 46
viii
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
28 Overview of 3GPP Releases 1329 Technical Overview of LTE 15210 Architecture of LTE 16
2101 Core Network 162102 Radio Access Network 17
3 Design and Implementation 2031 LTE Network Performance Evaluation with Generated Traffic 20
311 Experimental Procedure 21312 Gateway 22313 Sender 22314 Receiver 22315 Measurement Point (MP) 22316 Consumer 22
32 Measurements 23321 One Way Delay (OWD) 23322 Packet Loss (PL) 23323 Inter Packet Delay (IPD) 23
33 Video Quality Assessment using SubjectiveAnalysis 24331 Experimental Setup and Procedure 24332 WOWZA Media Server 25333 Test Video Parameters 25334 NetEm 26335 Delay 27336 Packet Loss 27337 Assessment of Videos 27
4 Results and Analysis 3041 Analysis of LTE Network Performance with Generated Traffic 30
411 One Way Delay for UDP packets in Uplink 304111 Minimum One Way Delay 304112 Maximum One Way Delay 314113 Mean One Way Delay 31
412 Inter Packet Delay for UDP packets in Uplink 324121 Minimum Inter Packet Delay 324122 Maximum Inter Packet Delay 334123 Mean Inter Packet Delay 34
413 Packet Loss for UDP in Uplink 34414 One Way Delay for TCP packets in Uplink 35
4141 Minimum One Way Delay 354142 Maximum One Way Delay 354143 Mean One Way Delay 36
415 Inter Packet Delay for TCP packets in Uplink 37
iv
4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38
416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41
451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45
452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49
5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52
Bibliography 53
v
List of Figures
21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18
31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27
41 Minimum One way Delay for Generated UDP Packets forUplink 30
42 Maximum One way Delay for Generated UDP Packets forUplink 31
43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for
Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for
Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for
Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for
Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for
Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-
ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet
size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43
vi
418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49
vii
List of Tables
21 Five-level scale for rating overall quality of video 13
31 Test Video Parameters 26
41 MOS for Delay Variation 4342 MOS for Packet Loss 46
viii
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
4151 Minimum Inter Packet Delay 374152 Maximum Inter Packet Delay 384153 Mean Inter Packet Delay 38
416 Packet Loss for TCP in Uplink 3942 Gateway Evaluation 3943 One Way Delay Comparison in TCP and UDP 4044 One Way Delay for Video Streaming Over LTE 4045 QoE Analysis of Video Streaming 41
451 Packet Delay Variation 424511 Packet Delay Variation for TCP 424512 Packet Delay Variation for UDP 444513 Standard Deviation for Delay Variation 454514 Confidence Interval for Delay Variation 45
452 Packet Loss 464521 Packet Loss for TCP 464522 Packet Loss for UDP 474523 Standard Deviation for Packet Loss 484524 Confidence Interval for Packet Loss 49
5 Conclusion and Future Work 5151 Conclusion 5152 Future Work 52
Bibliography 53
v
List of Figures
21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18
31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27
41 Minimum One way Delay for Generated UDP Packets forUplink 30
42 Maximum One way Delay for Generated UDP Packets forUplink 31
43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for
Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for
Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for
Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for
Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for
Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-
ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet
size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43
vi
418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49
vii
List of Tables
21 Five-level scale for rating overall quality of video 13
31 Test Video Parameters 26
41 MOS for Delay Variation 4342 MOS for Packet Loss 46
viii
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
List of Figures
21 Quality of Experience Measurement 1022 Evolution of LTE 1523 LTE Radio Access Network 18
31 Detailed Experimental Set up 2132 Experimental Set up for Video Streaming 2533 Screen Shot of QoE Evaluation Tool 27
41 Minimum One way Delay for Generated UDP Packets forUplink 30
42 Maximum One way Delay for Generated UDP Packets forUplink 31
43 Mean One way Delay for Generated UDP Packets for Uplink 3244 Minimum Inter Packet Delay for Generated UDP Packets for
Uplink 3345 Maximum Inter Packet Delay for Generated UDP Packets for
Uplink 3346 Mean Inter Packet Delay for Generated UDP Packets for Uplink 3447 Packet Loss for Generated UDP Packets for Uplink 3448 Minimum One way Delay for Generated TCP Packets for Uplink 3549 Maximum One way Delay for Generated TCP Packets for
Uplink 36410 Mean One way Delay for Generated TCP Packets for Uplink 36411 Minimum Inter Packet Delay for Generated TCP Packets for
Uplink 37412 Maximum Inter Packet Delay for Generated TCP Packets for
Uplink 38413 Mean Inter Packet Delay for Generated TCP Packets for Uplink 38414 Packet Loss for Generated TCP Packets for Uplink 39415 Minimum One Way Delay for Generated TCP and UDP pack-
ets for Uplink 40416 Minimum One Way Delay for Generated TCP and UDP packet
size of 1500 bytes for Uplink 41417 MOS for TCP videos subjected to Delay Variation 43
vi
418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49
vii
List of Tables
21 Five-level scale for rating overall quality of video 13
31 Test Video Parameters 26
41 MOS for Delay Variation 4342 MOS for Packet Loss 46
viii
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
418 MOS for UDP videos subjected to Delay Variation 44419 Standard Deviation for Delay Variation 45420 Confidence Interval for Delay Variation 46421 MOS for TCP videos subjected to Packet loss 47422 MOS for UDP videos subjected to Packet Loss 47423 Standard Deviation for Packet Loss 48424 Confidence Interval for Packet Loss 49
vii
List of Tables
21 Five-level scale for rating overall quality of video 13
31 Test Video Parameters 26
41 MOS for Delay Variation 4342 MOS for Packet Loss 46
viii
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
List of Tables
21 Five-level scale for rating overall quality of video 13
31 Test Video Parameters 26
41 MOS for Delay Variation 4342 MOS for Packet Loss 46
viii
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Acronyms
1 G 1st Generation
2 G 2nd Generation
3 G 3rd Generation
4 G 4th Generation
AVC Advance Video Codec
BTH Blekinge Tekniska Hogskolan
CI Confidence Interval
CN Core Network
CAGR Compound Annual Growth Rate
DAG Digital Acquisition and Generation
DPMI Distributed Passive Measurement Infrastructure
EDGE Enhanced Data rates for Global Evolution
EPS Evolved Packet System
ETSI European Telecommunications Standards Institute
FPS Frames Per Second
FR Full-Reference
GPRS General Packet Radio Services
GPS Global Positioning System
GSM Global System for Mobile Communications
HD High Definition
HSDPA High-Speed Downlink Packet Access
ix
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
HSS Home Subscriber Server
HSUPA High Speed Uplink Packet Access
HTML Hyper Text Markup Language
IMS IP Multimedia Subsystem
IP Internet Protocol
IPD Inter Packet Delay
IPTV Internet Protocol Tele Vision
ISO International Organization for Standard
ITU International Telecommunications Union
ITU-R International Telecommunication Union Radio CommunicationSector
ITU-T International Telecommunication Union Telecommunication Stan-dardization Sector
KBPS Kilobits Per Second
KPI Key Performance Indicator
LTE Long Term Evolution
MArC Measurement Area Controller
MBMS Multimedia Broadcast Multicast Services
MIMO Multiple-Input Multiple-Output
MME Mobility Management Entity
MMS Multimedia Messaging Support
MOS Mean Opinion Score
MP Measurement Point
MPEG Moving Pictures Expert Group
MSE Mean Squared Error
MTC Machine Type Communication
MTU Maximum Transmission Unit
NTP Network Time Protocol
x
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
OFDMA Orthogonal Frequency-Division Multiple Access
OWD One Way Delay
PCRF Policy Control and Charging Rules Function
P-GW PDN Gate Way
PDN Packet Data Network
PDV Packet Delay Variation
PEVQ Perceptual Evaluation Video Quality
PL Packet Loss
PSNR Peak Signal-to-Noise Ratio
QoE Quality of Experience
QoS Quality of Service
RAN Radio Access Network
RR Reduced Reference
RTMP Real Time Messaging Protocol
RTP Real-Time Transport Protocol
RTSP Real Time Streaming Protocol
SAE System Architecture Evolution
SC-FDMA Single-Carrier Frequency-Division Multiple Access
SD Standard Deviation
S-GW Serving Gateway
TCP Transport Control Protocol
TS Traffic Shaper
UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System
UR User Rating
UE User Equipment
USB Universal Serial Bus
xi
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
UTRAN Universal Terrestrial Radio Access Network
VGA Video Graphics Array
VoD Video on Demand
VoIP Voice Over Internet Protocol
WCDMA Wideband Code Division Multiple Access
WMS Wowza Media Server
xii
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Introduction
1
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Chapter 1
Introduction
For the last two decades radio access technologies are not just limited toprovide voice communications alone but also used for the video and dataapplications as well Due to the rapid development of technology used intelecommunication systems and consumer electronics network operators arenow able to provide better Internet services over radio networks After thedevelopment of 2G and 3G technologies the Internet based services areavailable on mobile systems namely mobile broadband
According to CISCO report mobile video has been growing at a Com-pound Annual Growth Rate (CAGR) of 75 percentage between 2012 and2017 and it is the highest growth rate of any other mobile application [1]Meeting this demand maintaining the Quality of Service and user satis-faction has become a big challenge to network operators To achieve thisa radio interface is needed which can provide the best Quality of Serviceswith design parameters like Data rates Delay and Capacity
One of the main reasons for LTE evolution is to provide the IP basedservices to people on mobile devices with better QoS [2] Some services havebeen already provided by 3G networks but providing HD video streaminginteractive video gaming and other multimedia services without degradingthe Quality of Services is a major challenge Among these multimedia ser-vices video streaming over mobile Internet is the most popular one In theburst growth of data rates and services being aware of user experience isimportant to maintain the service quality and the application performance
The Quality of Experience is defined as ldquoThe process of understand-ing the actual performance of services as delivered to the customer forthe purpose of ensuring those services meet customer expectations and re-quirementsrdquo Understanding user experience is very critical for the networkoperators in managing the QoS of the network Quality of Experience mea-surements are made at the point of delivery directly from the subscriberrsquossmart phone or PC QoE measurements deals with how well applications(Video streaming VOIP and Web browsing) work in the hands of subscriber
2
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 1 INTRODUCTION 3
[3] QoE measurements require an understanding of the Key PerformanceIndicators (KPI) that impact on the user perception KPIrsquos vary with theservice type and services like VoIP Video streaming On-line gaming In-ternet browsing has unique performance indicators to measure [4] QoEconsiders the individual subscriber experience with a service unlike networkconditions in QoS The knowledge of user actual experience is very impor-tant for the operators to know the customers satisfactory levels and thenoperators can concentrate on issues to prevent the churn
The mobile communication technologies are tracked back to various gen-erations 1G 2G 3G and 4G 1G which started in 1980 stands for first gener-ation of wireless telecommunications popularly known as Cellular phones inwhich analog radio signals are used In 1991 2G (Second Generation) wire-less telephone technology started using digital mobile systems [2] The sec-ond generation mobile technologies provide low bandwidth services whichare suitable for voice traffic The packet data over cellular systems startedwith the introduction of GPRS (General Packet Radio Services) in GSM(Global System for Mobile Communications) With the introduction of 3Gtechnology network operators can provide better and more advanced ser-vices like video calls and mobile broadband services
In our thesis we worked on user quality assessment for video streamingover LTE network We report on user perception of video quality degrada-tion of selected videos which are subjected to different delays and packetlosses This thesis is specifically aimed to understand the experience of HighDefinition videos with H264AVC We report the user experience by con-ducting Subjective Assessment as per the International TelecommunicationsUnion (ITU) [5]
In addition to this we also studied the Uplink behaviour of the LTEnetwork by calculating the One-way Delay Inter Packet Delay and Packetlosses with different payloads at different data rates We analyzed the Qual-ity of Service of video packets over LTE network with OWD IPD and Packetlosses as QoS metrics
11 Motivation
LTE is the latest technology in the telecommunications world using whichnetwork operators are able to provide advanced multimedia applications tousers maintaining Quality of service [2]
By the introduction of LTE network users are able to enjoy the broad-band quality Internet services [2] on the mobile devices but providing theadvanced multimedia services over radio network without degrading theQuality of Services is a big challenge Video streaming is the most pop-ular application in the next generation mobile systems Due to the rapidincrease in data rates using LTE technology users are able to watch High
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 1 INTRODUCTION 4
Definition videos on the mobile devices and at the same time the mobilesystems are being manufactured to access advanced services
In this current day of huge resource demand applications understandingthe user experience is very critical for the network operators to manage theQoS of the network and hence our motivation to measure and analyze itWe choose High Definition videos with 1280 times 720p and H264AVC sinceH264 codec is the widely used codec for video streaming As comparedto the previous codec like MPEG-2 and MPEG-4 Visual AVC providesgood quality of video for wide range of services like broadcast multimediastreaming and Video on Demand
The QoS and QoE are so interdependent and they have to be studied andmanaged with common understanding So as a part of QoS evaluation weconducted experiments to measure the OWD IPD and packet loss for gener-ated TCP UDP packets over LTE network for Uplink We conducted theseexperiments to know the behaviour of TCP and UDP packets for differentdata rates with different payloads We chose uplink since video sharingbecame popular with the emerging of websites like YouTube Facebook andVimeo According to YouTube statistics 72 hours of video are uploaded toYouTube every minute [6]
12 Related Works
In paper [7] the authors proposed QoE assessment models for video stream-ing services like IPTV by using QoS parameters in network layer This helpsthe network service provider in providing multimedia services with improvedQoE by using the suggested QoE assessment process This also helps thenetwork service provider to prevent the unnecessary expenses for mainte-nance and repair of the network
In paper [8] the authors evaluated the QoE measurement in a live 3Gnetwork with automatic data capturing tools are used in the experimentEvaluation is carried out by subjective methods relevant to a set of objectiveparameters The author concluded that the QoE of mobile video streamingwas influenced by the QoS and with the context also
In paper [9] the authors investigated the QoE evaluation method of videostreaming service in 3G networks The author suggested a non reference QoEmodel of video streaming services based on the gradient boosting machine
In paper [10] the authors analyzed the perception of users towards thevideos encoded with H264 baseline profile in laptops and mobile devicesThe videos are streamed through an emulated network with packet lossesand packet delay variations The obtained results from both devices arecompared using matched-sample-test The conclusion infers that the devicedoes not show any impact on user perception for videos of same resolution
In paper [11] the authors investigated on different types of QoE analysis
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 1 INTRODUCTION 5
and proposed a flexible framework well capable to correlate with both packetloss ratio and subjective quality degradation The proposed metric and theframework provide help for performing easy and accurate QoE evaluationson streaming applications
In paper [12] the authors presented a new model for non-intrusive pre-diction of H264 encoded video quality over UMTS networks The test bedwas evaluated on the NS2 based UMTS simulation network The proposedmodel was based on combination of a set of objective parameters in thephysical and network layers in terms of Mean Opinion Score (MOS)
In paper [13] the author proposed a conceptual model of QoE cor-responding to the hourglass model of Internet architecture based on thestreaming video quality of experience and its factors This model of QoEhas four layers similar to the Internet hourglass model and each layer hasa role towards the user perceived quality
In paper [14] the author analyses the user perception towards the QoEof video which is encoded with H264 baseline profile and streamed throughan emulated network with packet loss and packet delay variation The ex-periment was conducted both on a laptop and a mobile device
In paper [15] the authors analyzed the effect of QoS parameters likeend-to-end delay packet loss and packet delay on the performance of videoconferencing in the LTE network The authors carried out their experimenton OPNET 160 [16] simulator The results are evaluated by taking threenetwork scenarios namely low load medium load and high load
In paper [17] the authors analyzed the impact of payload size and datarate on one-way delay and packet loss in network on three different com-mercial 3G mobile operators available in Sweden The measurement in thenetwork are carried out by using Endace DAG [18] cards and the EndaceDAG are synchronized with GPS to get an accurate measurement
In paper [19] the authors analyzed the one-way delay in wireless broad-band network based on traffic measurements The experimental setup isdesigned in such a way to get perfect time synchronization and accurateresults The authors concluded that the one-way delay of uplink is muchhigher than the one-way delay of downlink
In paper [20] the authors investigated the operator services on one-waydelay and jitter using packets of different protocols with random packetsize and random IPD They investigated the impact of constant IPD andreducing the interval of IPD on OWD in 3G networks They also investigatedthe one-way delay for Constant Bit Rate (CBR) and Variable Bit Rate(VBR) transmission patterns
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 1 INTRODUCTION 6
13 Contribution
The experiments were conducted in two phases In the first phase we con-ducted the video quality assessment using subjective method Videos aretransmitted over LTE network and subjected to different delays and packetlosses There has been previous works [21] [22] done on video streamingover 3G networks and other wireless networks However most of the worksare based on simulations and objective analysis using Peak Signal-to-NoiseRatio (PSNR) and Perceptual Evaluation of Video Quality (PEVQ) toolsPEVQ tool is recommended by ITU but it has limitation of video length notmore than 20 seconds [23] We adopted subjective analysis method whichgives better user opinion than objective analysis In wireless radio networksit is hard to predict the network behaviour with less than one minute video[24] So we choose the videos which are having a duration of four minutes
The Quality of end user experience affected by the technical factors ofQoS (network quality and network coverage) and non technical Subjectivefactors (service content ease of service setup and pricing) So we attemptedto know the performance of LTE network by measuring QoS metrics thatare OWD IPD and Packet loss for uplink We measured these metrics onan experimental test bed where OWD is calculated with the packet tracescollected at the link level It gives the possible precise measurements up to60 nano seconds [25] With the collected traces we calculated the OWDIPD and Packet losses using Perl program
14 Aims and Objectives
This Thesis aims at studying the user experience on video quality withthe parameters packet loss and delaydelay variance over LTE network andH264 as video codec Another aim is to know the LTE network behaviourin terms of OWD IPD and Packet loss for generated traffic on UDP andTCP protocols
The objectives are as follows
bull To understand the user perception of video quality for high definitionvideos with packet losses and delay variations over LTE network withH264 as codec
bull To understand the user perception of video quality for different pro-tocols
bull To understand the LTE network performance in terms of OWD IPDand Packet loss at different data rates for different payloads
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 1 INTRODUCTION 7
15 Research Questions
1 What is the effect of TCP and UDP protocols on OWD IPD andPacket loss in LTE network uplink with different payloads at differentdata rates
2 How is the user perception affected by the delay variation in videostreaming over LTE network using TCP and UDP protocol
3 How is the user perception affected by the packet loss in video stream-ing over LTE network using TCP and UDP protocol
16 Thesis Outline
The rest of the document is as follows Chapter 2 provides the technicalbackground of Quality of Experience and LTE releases Chapter 3 describesthe experimental methodology design and implementation Chapter 4 illus-trates the results and its analysis Chapter 5 comprises the conclusion andfuture work
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Technical Background
8
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Chapter 2
Technical Background
21 Quality of Experience
QoE is also defined [23] as ldquoThe overall acceptability of an application orservice as perceived subjectively by the end userrdquo It mainly deals withhow an individual is satisfied with the provided service in terms of usabilityaccessibility retain ability and integrity of the service Quality of Experienceconsiders the complete end-to-end system effects like the effects of the clientnetwork and infrastructure of the services It also takes into considerationthe end userrsquos mood emotions and physical status which encompasses thepsychology of the end user So there is no particular defined statement forQoE that is accepted universally [26]
QoE considers the individual subscriber experience with a service unlikenetwork conditions in QoS The knowledge of actual user experience is veryimportant for the operators to meet the customerrsquos satisfactory levels Indoing so operators can turn their attention on issues to prevent the churn
22 Video Streaming
Today video sharing is the most effective way of entertainment and gainingknowledge In order to share a video it must be stored and transmitted overa communication channel but it is expensive to transmit a raw video overa communication channel because of the huge amount of data size Evento store a raw video it requires a lot of space on the data storing devicesUsually the video taken from camera footage contains lot of redundant dataSo there is a need to reduce the size of the redundant data in the raw videoalso considering the quality of the video Here video compression comes intothe picture to reduce the redundant data by considering the quality of thevideo [27]
9
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 2 TECHNICAL BACKGROUND 10
Figure 21 Quality of Experience Measurement
23 Video Compression
Video Compression is a technique where the raw video is compressed usingmathematical models and algorithms to reduce the size of the video to lowerbit rates Video compression is an essential process for video applicationslike Internet video streaming mobile TV video conferencing digital televi-sion and DVD-Video [28] Video compression is mainly classified into twotypes lossless compression and lossy compression In lossless compression noinformation is lost in the compression process So it is possible to recover theoriginal raw video from a compressed video In practical cases the amountof data reduced is less Quality of the video is maintained well in losslesscompression [29] This type of video compression is not used for streamingvideos because even though video is compressed it still maintains a largedata size
In lossy compression as the name suggest information is lost in compres-sion process The information once lost cannot be retrieved [30] Obviouslythe size of the video is reduced and the quality of video is degraded tooThis technique is predominantly used for video streaming services Eventhough the quality of the video is lost it is still perceivable Video compres-
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 2 TECHNICAL BACKGROUND 11
sion should be done at an optimum level while simultaneously consideringthe video quality
24 Video Format
The video format consists of two distinct technology concepts Video Codecand Video Container
241 Video Codec
Video compression process involves a complementary pair of systems a com-pressor (encoder) which converts the source video into compressed formbefore the transmission or storage and a decompressor (decoder) which con-verts the compressed data back to represents the original data So thisencoder-decoder pair is called as CODEC Video codec is a software pro-gram that compresses a raw video into a compressed video of small datasize in a way that the compressed video must play on the computer sinceraw or uncompressed data requires a large bit rate approximately 216 MBper second [28] Video codec can only compress a video similarly audiocodec is used for audio file and played along with video files
Different types of codes are available to compress raw video and theselection of video codec depends on various factors like size of video typeof video streaming and type of application [31] Nowadays there are manyvideo codes available for video compression some of them are MPEG-1MPEG-2 MPEG-3 MPEG-4 H264 and Vorbis
Among all available video codes H264 is the most widely used codecMain features of this codec are providing better video quality at lower bit-rates fast encoding speed and other advanced features make H264AVCbetter than its counterparts [28]
242 Video Container
Video Container describes the structure of the video in which way it hasbeen compressed and stored [32] Video codec compresses the raw video intoa format which is considered as the video container Examples of the videocontainers are avi mp4 mov and asf
243 H264AVC Codec
H264 is a method and format for video compression It was developed basedon the concepts of earlier standards such as MPEG-2 and MPEG-4 Visualand provides better compression efficiency [28] It also provides features likebetter-quality compressed video and greater flexibility in transmitting andstoring videos Furthermore it offers robust compression for a wide range of
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 2 TECHNICAL BACKGROUND 12
applications from low bit rate mobile video applications to high definitionbroadcast services
25 Types of Video Streaming
Nowa-days different types of video streaming applications are in use Someof them are point-to-point communication broadcast communication andmulticast communication All these video streaming applications are mostlydepends on two video streaming types One of them is live video streamingwhich is a process of real time transmission of a video over Internet [33] Sothe streamed video file can be viewed on smart phones mobiles and personalcomputers The video application is mainly used in live sports broadcasting
Another type of video streaming is Video-on-Demand this video stream-ing process is quite contrary to the previous type In this video streamingthe selected video file is played whenever the viewer wishes to watch thevideo In this type of streaming one-to-many viewers can view the sameor different video [34] This process is mainly observed in video streamingwebsites like YouTube Hulu and DailyMotion
26 Supported Protocols for Video Streaming
There are several protocols that support media streaming Some of the ma-jor protocols are Hypertext Transfer Protocol (HTTP) Session DescriptionProtocol (SDP) Real-time Streaming Protocol (RTSP) Real-time Trans-port Protocol (RTP) Real Data Transport (RDT) and Real-time TransportControl Protocol (RTCP) [35] Each protocol is used depending on the typeof application in that particular point and also depends upon the require-ment of service For most of the video streaming protocols TCP and UDPserves as the underlying protocol UDP is a preferable to its contrary pro-tocol TCP in video streaming application because UDP send packets at aconstant rate and it doesnrsquot care about the lost packets But TCP is alsowidely used in video streaming because of benefits of streaming with TCPincluding its retransmission capabilities congestion control and flow control[36] On the same hand TCP introduces delay due to the retransmissionof data whenever data is lost But in case of UDP there is no point ofretransmission
27 Assessment of Videos
When it comes to the point of video quality assessment there are mainlytwo types of methods They are as follows
bull Objective Assessment
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 2 TECHNICAL BACKGROUND 13
bull Subjective Assessment
The Objective video quality assessment method is based on Mathemati-cal models and fast algorithm that produces the results approximately equalsto the subjective video quality assessment and it does not involve any hu-man grading It is a software program designed to deliver results based onerror signal ratio of the original and processed video The most popularObjective methods are Mean Squared Error (MSE) Perceptual Evaluationof Video Quality (PEVQ) and the Peak Signal -to- Noise Ratio (PSNR) [37][38] This method has few categories referred as Full-Reference (FR) andReduced - Reference (RR) video quality metrics [39]
The other video assessment includes Subjective analysis which is basedon human perception The subjective video quality assessment is consideredas accurate way of measuring quality of video compared to objective assess-ment For subjective video quality assessment a set of videos are given tothe subjects for rating the videos on a scale of five (5) and the grades forquality is given in Table 21 The rating given by the subjects are known asMean opinion Score (MOS) The subjects include experts and non- expertobservers
MOS Rating User Opinion
5 Imperceptible4 Perceptible but not annoying3 Slightly annoying2 Annoying1 Very annoying
Table 21 Five-level scale for rating overall quality of video
28 Overview of 3GPP Releases
The modern society is becoming rapidly dependant on high speed mobilenetworks for instant access to information The user needs to have instantaccess to information thereby facilities a way for the creation of user ap-plications which required low jitter and low latency Usage of applicationslike banking multiplayer on-line gaming downloading music watching livenews and sports IPTV and so on over the Internet is rapidly increasingThere is a great and tremendous need for high speed data networks requiredto meet the above user applications On this behalf 3rd Generation Part-nership Project (3GPP) developed LTE to have higher data rate 3GPPis a collaboration agreement that was established in December 1998 and itwas formed by six telecommunication standards that came together on anagreement from different countries known as the Organizational Partners
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 2 TECHNICAL BACKGROUND 14
3GPP has developed different mobile technologies and those technologiesare differentiated by releases A brief overview of different 3GPP releasesfrom GSM to LTE-Advanced is as follows
In Releases 99 [40] the GSM specifications of the Universal TerrestrialRadio Access Network (UTRAN) are developed UMTS was first standard-ized by the European Telecommunications Standards Institute (ETSI) inJanuary 1998 Mobile technologies like EDGE (Enhanced Data rates forGSM Evolution) provides high data rate than GPRS which offers serviceslike messaging email etc Majority of technologies in usage are based onrelease 99
Release 4 [41] is provided with minor improvements in UMTS networkIt mainly concentrates on Multimedia Messaging support which includesdevelopment of the MMS Reference Architecture MMS service featuresstreaming Support in MMS and a lot more
In release 5 [42] the 3GPP featured a new technology called HSPDAwhich helps the wireless operators to provide the customer need of highspeed wireless data services with improved spectral efficiency In the samerelease it also introduced the IP Multimedia Subsystem (IMS) architectureIMS enhances the end user experience for multimedia application and alsofacilitates the use of IP (Internet Protocol) for packet communications in allwireless networks [43]
Release 6 [44] is mainly concentrated on Quality of Service (QoS) formultimedia applications Enhanced Multimedia BroadcastMulticast Ser-vices (MBMS) which is a user service enables data to deliver to a set ofusers using same radio resources within a service area like High Speed UplinkPacket Access (HSUPA) for high uplink speed
Release 7 [45] focuses on decreasing latency improved QoS for the real-time applications such as gaming VoIP In this release the spectral efficiencyof the HSPA is increased by the introduction of Multiple-Input Multiple-Output (MIMO) antenna systems Enhancements to GSM with EvolvedEDGE which increases usual throughput rates reduces the latency by twoand improves spectral efficiency
Release 8 [46] consists of evolution of HSPA features such as 64 QAMand simultaneous use of MIMO Innovation of LTE technology which is ex-pected to meet the high data rate requirements of the end user Reduceduser plane latency resulting highly improved user experience with full mo-bility Using single-carrier frequency domain multiple access (SC-FDMA)for uplink and orthogonal frequency domain multiple access (OFDMA) fordownlink It introduces Evolved Packet System (EPS) to provide networkarchitecture which integrate common mobility security and QoS mecha-nisms for fixed and mobile broadband accesses
Release 9 [47] provides much more enhancement to both HSPA andLTE by including HSPA dual-carrier operation in combination with MIMOEvolution of the IMS architecture introduces the concept of LTE Femto
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 2 TECHNICAL BACKGROUND 15
cells It also initiated the study of Advanced LTEIn Release 10 [48] new concepts in LTE-Advanced are introduced like
Carrier Aggregation (CA) multi-antenna enhancements and relays It alsoincludes Self Organizing Networks (SON) Heterogeneous Networks (Het-Nets) quad-carrier operation for HSPA+ etc Enhanced uplink and down-link schemes for much more higher data rates than LTE-Advanced
In Release 11 [48] will build on the advancements and refinements ofcapabilities of technologies that are developed in release 10 Enhancementsto relays Carrier Aggregation and MIMO etc
Release 12 [48] includes the study of network optimization for MachineType Communication (MTC) Nonvoice emergency services and Session Ini-tiation Protocol Uniform Resource Identifier (SIP URI) portability
Figure 22 Evolution of LTE
29 Technical Overview of LTE
The term LTE includes the development of the Universal Mobile Telecom-munications System (UMTS) radio access through the Evolved UTRAN (E-UTRAN) It is mainly accomplished by the evolution of core network knownas System Architecture Evolution (SAE) The developed new architectureprovides higher rate data delivering capacity to the LTE network By thisLTE is able to support different types of services including HD video stream-ing VoIP Multi user online gaming Video on demand Push-to talk andPush-to-view The main features and capabilities of LTE [49] [50] [51]are
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 2 TECHNICAL BACKGROUND 16
bull The downlink speed is up to 100 Mbps and the technique used isOrthogonal frequency-division multiplexing (OFDM)
bull The uplink speed is up to 50 Mbps and the technique used is Single-carrier frequency-division multiple access (SC-FDMA)
bull It uses 4x4 antennas for downloading rates and it uses a single antennafor uploading rates
bull The data transmission in LTE is significantly low for application thatuse low latency such as VoIP Online Multi player gaming etc
bull The user plane latency offered in LTE is less than 10 ms and thecontrol plane latency is less than 100 ms
bull In the case of mobility LTE supports up to 500 kmph but like othermobile technologies it will be optimized for lower speed from 0 to 15kmh
bull LTE supports higher flexibility in carrier bandwidths the set of band-widths actually supported are 125 25 5 10 15 and 20 MHz
bull LTE is capable of delivering optimum performance in a cell size upto 5 km but it still can deliver its effective performance up to 30 kmWhereas with its limited performance it can deliver up to 100 kmradius
210 Architecture of LTE
The enhancement features like higher packet data rates and significantlylower-latency of LTE cannot be possible without the evolution of System Ar-chitecture Evolution (SAE) This includes the Evolved Packet Core (EPC)network The Evolved Packet System (EPS) consists of LTE and SAE Itwas decided to have a rdquoFlat Architecturerdquo EPS [52] is defined to supportonly packet-switched traffic It uses the concept of EPS bearers to direct theIP traffic from a gateway in the Packet Data Network (PDN) to the UserEquipment (UE) EPS bearer is a virtual connection provides transport ser-vice with specific QoS attributes between the gateway and the UE
Likewise the HSPA architecture the LTE architecture also divided intotwo networks as radio access network and a core network However theultimate goal of the LTE is to minimize the number of nodes As a resultof this the Radio Access Network (RAN) contains only one node
2101 Core Network
The nodes contained in the core network are explained below [53] [54]
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 2 TECHNICAL BACKGROUND 17
Policy Control and Charging Rules Function (PCRF) PCRFprovides the service management and control of the LTE services It isresponsible for policy control decision making besides regulating the flowbased charging functionalities in the Control Enforcement Function (PCEF)It helps to have dynamic control over QoS in turn help operators to providecustomers with a variety of QoS and charging options when opting for aservice
Home Subscriber Server (HSS) HSS is the master database it con-tains the user subscription data to support call control and session manage-ment entities This includes the subscribed QoS profile and restrictions toaccess when the subscriber is in roaming It provides the authenticationand authorization of subscribers It holds the information related to thelocation of subscriber and the information about the Packet Data Network(PDN) to which the subscriber is connected HSS also supports multi-accessmulti domain networks which provides end to end traffic handling facilitiesfor the subscribers moving between LTE and Wireless Local Area Network(WLAN)
PDN Gateway (P-GW) P-GW is responsible for allocating the dy-namic IP addresses to the subscriber and routes the user plane packets Itprovides the QoS enforcement and flow based charging as per the policiesin the PCRF PCRF gives the instructions on how to deal with a particu-lar service data flow by keeping in mind the QoS terms of priority as perthe subscriber profile It acts as an anchor for mobility between 3GPP andnon-3GPPP technologies
Serving Gateway (S-GW) S-GW directs data packets to all sub-scribers which acts as a local mobility anchor for data bearer that movesbetween eNB hand overs It also acts as the anchor between LTE and 3GPPtechnologies It gets the information about the bearer when the UE is inan idle state S-GW terminates the Downlink data path and also triggerspaging when DL data arrives for the UE
Mobility Management Entity (MME) MME is the key controlnode for LTE access network that processes the signaling between UE andthe Core network It is responsible for choosing the SWG for a UE at theinitial registration process and also for intra-LTE handover including CoreNetwork (CN) node relocation The main functions that are carried by theMME are related to the bearer management and connection managementBearer management includes maintaining establishment and release of thebearers And the connection management includes establishment of theconnection as well as security between the network and UE
2102 Radio Access Network
The Evolved UMTS Terrestrial Radio Access Network (E-UTRAN) man-ages the radio communication between the User Equipment and the Evolved
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 2 TECHNICAL BACKGROUND 18
Packet Core by using one component called eNodeB Unlike normal NodeBeNodeB does not have centralized controller system and hence the E-UTRANis regarded as a flat architecture eNodeB is responsible for Radio ResourceManagement which includes radio bearer control scheduling and dynamicallocation of links to UE for uplink and downlink Also it compresses theIP packet header for efficient use of resources Encrypted data is sent overthe radio interface for better security
eNodeBrsquos are connected to EPC by means of S1 interface more specifi-cally to S-GW by the S1 User plane part and to MME by the means of S1control plane part ENodeBrsquos are interconnected by means of X2 interfaces[55]
Figure 23 LTE Radio Access Network
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Design and Implementation
19
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Chapter 3
Design and Implementation
This section describes the hardware utilities and software tools that we usedfor conducting the experiments The section consists of explanation of twoexperimental setups one for the evaluation of LTE network performancewith OWD IPD and Packet loss as QoS metrics using generated traffic foruplink and the other for video quality assessment over LTE network usingsubjective analysis
Before proceeding to our aim of QoE evaluation of video streaming overLTE network we measured the QoS KPIrsquos of live LTE network BecauseQoE and QoS are integral part of each other and the evaluation of QoEwould not be complete without addressing the QoS [56]
31 LTE Network Performance Evaluation with Gen-erated Traffic
Quality of Service is basically technical concept and it is expressed measuredand understood in networks and network elements which usually has littleconcerned to user [56]
In this section we used Distributive Passive Measurement Infrastructure(DPMI) in our experimentation which is a dedicated hardware to performmeasurements at the link level to evaluate the performance of LTE networkin terms of OWD IPD and Packet loss for Uplink The traffic generatingtool is used to generate the TCP and UDP packets from sender system (S)to receiver system (R) The test bed is configured in such a way that thesender is connected to LTE network using Huawei E398 LTE USB modemhaving business type subscription and the receiver is connected to the BTHInternet We configured our setup in such a way that the sender is able tosend the TCP and UDP packets over LTE network and the receiver is ableto receive those packets via BTH Internet In between sender and receiverwe placed Measurement Point (MP) to capture the traffic This MP givesthe accurate time stamp of packets when it leaves from sender and arrives
20
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 3 DESIGN AND IMPLEMENTATION 21
at the receiver
311 Experimental Procedure
The detailed experimental setup is shown in Figure 31 The packet flowin this experiment starts from the sender system which generates the UDPpackets with the help of Traffic generator and it passes to Gateway ThisGateway sends packets to receiver through LTE network The receiver sys-tem connected to BTH Internet receives the packets The transmitted trafficat the sender and receiver end is fed to the MP through wiretaps which areconnected to it by monitoring ports The time stamping of packets at MPdoes not effect the actual packet flow since the wiretaps duplicates thepackets without disturbing the original packets
Figure 31 Detailed Experimental Set up
The traffic generator tool consists of two programs one is client programand the other is server program The client program is installed in sendersystem and the server program is installed in receiver system The clientprogram consists [20] of Experiment number (E) Run Id (R) Key Id (K)destination IP destination port number number of packets to be sent lengthof packets and inter frame gap in micro seconds The server program consistsof Experiment number (E) Run Id (R) Key Id (K) [20] The collected tracesat the consumer are analyzed using Perl program based on the sequencenumbers along with above mentioned E R K values respectively Thesevalues are used to recognize the packets at source and destination [20] Byanalyzing the collected traces we calculate the minimum maximum andmean of OWD IPD and packet loss percentage for different payloads atdifferent data rates Based on the calculated values we plotted the graphs
The main components in this setup are Gateway Sender Receiver Mea-surement Point and Consumer
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 3 DESIGN AND IMPLEMENTATION 22
312 Gateway
Gateway is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM The gateway is connectedbetween sender and receiver as shown in Figure 31 The gateway is di-rectly connected to the sender with Ethernet cable via Broadcom Ethernetcard and the LTE USB modem is connected to the gateway We used thisgateway to access the LTE network since the sender with this LTE USBmodem cannot be connected to the Measurement Point We generate thetraffic using existing traffic generators [20] at the sender system The gen-erated packets are sent to receiver through Internet via gateway and thereceiver also communicates in the same way
313 Sender
Sender is a Linux based computer operating on Ubuntu 1204 OS consist-ing of AMD Athlon processor and 2 GB RAM It is connected directly tothe Gateway The sender generates the UDP and TCP traffic using trafficgenerator tool and it sends to receiver through Internet via gateway
314 Receiver
Receiver is a Linux based computer operating with Ubuntu 1204 OS con-sisting of AMD Athlon processor and 2 GB RAM It is connected to BTHnetwork using Ethernet It is used as a sink to receive the UDP and TCPtraffic transmitted from the sender system using traffic generator tool
315 Measurement Point (MP)
Measurement point (MP) is a Linux based system used for getting the times-tamps for the generated packets It is equipped with Endace DAG 35E cardsand these are enabled in such a way to capture the traffic without loss Itis synchronized to Network Time Protocol (NTP) server and GPS (GlobalPositioning System) to achieve the most possible accuracy in time stampingBy this we can achieve time stamp accuracy of 60 nanoseconds The MPfilters the packets according to the rules set by Marc [20] The wiretapsconnected at the sender and receiver ends are connected to the DAG cards
316 Consumer
Consumer is a Linux based computer operating with Ubuntu 1004 OS con-sisting of AMD Athlon processor 2 GB RAM It is installed with LibCa-putils It is used to get the link level packet traces which are broadcastedby the MP The collected traces are in cap format these are converted tohuman readable txt format using the LibCaputils
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 3 DESIGN AND IMPLEMENTATION 23
32 Measurements
The aim of our experimentation is to calculate the One-way delay (OWD)Inter-packet delay (IPD) and Packet loss over LTE network
321 One Way Delay (OWD)
One-way delay is the time taken by the packet to travel from source todestination In our case OWD of packet having the same sequence number(S) is calculated by subtracting the time stamp at receiver dag interface(T(S Dr)) with the time stamp at sender dag interface (T(S Ds))
OWD = T (SDr)minus T (SDs)
DrmdashDag at receiver sideDsmdashDag at sender side
In order to get fair one-way delay measurements the clocks should besynchronized Some of the available synchronization methods are NetworkTime Protocol (NTP) and Global Positioning System (GPS) By using NTPwe can able to synchronize the clocks within [10 ms 20 ms] for WAN scenar-ios and around 1 ms in LAN scenarios In our case for better synchroniza-tion we used DAG cards together with GPS which synchronizes the clock ofMPrsquos in order of 60 ns In practical case it is difficult to get high accuratetime stamp with two DAG cards (one at sender and other at receiver) Inorder to avoid this minor inaccuracy we used special wiring system wherewe capture traffic on singe DAG card and get time stamp from same clock[25]
322 Packet Loss (PL)
The percentage of packet loss is calculated as the ratio of the number ofpackets lost (L) to the total number of packets sent by the sender (N) Thenumber of lost packets is the difference in the number of packets receivedby receiver and the total number of packets sent by sender with unmatchedsequence numbers
PL = LN
323 Inter Packet Delay (IPD)
It is defined that the time interval between two successive packets at senderor receiver In our experiment we calculated at receiver end and it repre-sented by following equation
IPD = T (i+ 1)r minus T (i)r
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 3 DESIGN AND IMPLEMENTATION 24
TmdashTime of arrivalimdashith packetrmdashat receiver side
We calculated the IPD in these experiments at the receiver end andconsidering the no delay variation at the source side since we observed fromour traces that delay variation at source end showed a constant rate in mostcases We can also calculate the IPD at the source side but we need furtherstudy and moreover we are more concerned about the client side (receiver)
33 Video Quality Assessment using SubjectiveAnalysis
We streamed the videos without any delays and packet loss over LTE net-work and from the obtained videos we did not find significant freezes jerk-iness in them This might be the experiment was conducted in a region ofgood signal coverage Then arises the thought of conducting the experimentin the worst network performance conditions To achieve this traffic shaperis used to introduce different delays and packet loss in the video streaming
In this experiment two types of videos are taken and they are streamedover a live LTE network by introducing different delay variations and packetlosses The streamed videos are saved for video quality assessment
331 Experimental Setup and Procedure
The experiment is carried out using the experimental setup shown in Figure32 It contains the server which is operating on Ubuntu 1204 with AMDAthlon processor 2 GB RAM and it is connected to the BTH networkThe gateway is operating on Ubuntu 1204 with AMD Athlon processor2 GB RAM and it is connected to LTE network via USB modem Theclient runs on Ubuntu 1204 with AMD Athlon processor and 2 GB RAMIt is connected to gateway via Ethernet full duplex link bandwidth of 100Mbps The client system receives packets from server through LTE networkvia gateway The Measurement point is connected in between the sender andthe client The MP consisting of two DAG cards one of them is connectedto capture the traffic at the server side and the other DAG card is used tocapture the traffic at the client side The traces are collected and saved inthe consumer system
The Wowza Media Server software is installed in the server system andthe videos are encoded to the required format and placed in the server Inthis thesis the video streaming used is Video on Demand Before streamingthe video required delay and packet loss settings are made at traffic shaperwhich is installed on server system Videos are streamed from server to clientusing TCP and UDP protocols as per the request of client At client side
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 3 DESIGN AND IMPLEMENTATION 25
we used the VLC player to access the videos The VLC media player has afeature to save different videos that are being streaming over network Withthis option in VLC player we saved the streamed videos in the client systemThese streamed videos are saved in the client system The saved videos arefurther used to get the Mean Opinion Score from the users We performedour experiments in months of January February and March between 8 AMto 6 PM We repeated the experiments to provide reliability in results andto reduce the chance errors
Figure 32 Experimental Set up for Video Streaming
332 WOWZA Media Server
Wowza Media Server is a software used to stream video and audio files overpublic and private networks It can stream Live video streaming Videoon Demand and Video recording over Adobe Flash player Microsoft Sil-verlight player Apple iPhone iPad iPod touch Quick Time player Smartphones devices tablets and IPTV set-top boxes The supporting protocols ofWowza Media Server are Real Time Messaging Protocol (RTMP) MicrosoftSmooth Streaming Apple HTTP Live Streaming (HLS) Real Time Stream-ing Protocol (RTSP) Adobe HTTP Dynamic Streaming (HDS) Real-timeTransport Protocol (RTP) and MPEG-2 Transport Streams (MPEG-TS)[57] It is an alternative to Adobe Media Server Darwin Streaming ServerMicrosoft IIS Media Services and other media servers In our thesis we usedWowza media server and installed on server system the encoded videos aresaved in the server machine with a specific name These videos are accessedfrom client system using VLC player which is installed on it
333 Test Video Parameters
We selected two types of videos for our experimentation One is a Rugbygame video which comes under the fast moving video and the other is Big
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 3 DESIGN AND IMPLEMENTATION 26
Buck Bunny animated video which comes under slow moving video Wecovered the two types of videos suggested by ITU-T [58]
With the availability of high speed data networks users are more at-tracted to view high definition videos There is a scope in this direction totest the high definition videos over high speed data networks So we used720p High Definition videos with resolution 1280times 720 pixels and is codedwith a Main Profile encoder at Level 31 [28] These two raw videos usedfor video quality testing are taken from test media website [59] These twovideos are encoded to the required format using FFmpeg encoder [60]
The duration of selected video sequence is about 4 minutes In [24]performance of 3G data services was observed by taking packet data streamsfor 5 minutes To observe the user QoE for video streaming over radionetwork a minimum 4 minutes length of video is considered as appropriateWe chose the mp4 as a container since it is supported by most of the latestsmart phones based on Android and iPhone [61] [62]
We selected H264AVC as video codec because it has more advantagesthan MPEG-2 and MPEG-4 Visual in better image quality at same com-pressed bit rate [28] H264 offers greater flexibility in terms of transmissionsupport and compression options
Video sequences Rugby Big Bucks BunnyCodec Perceptible H264 Main Profile Level 31
Resolution 720p (1280times 720)Frame Rate 25fpsContainer MP4
Table 31 Test Video Parameters
334 NetEm
In our thesis we used NetEm traffic shaper for network emulation to varythe performance parameters like delay variation and packet loss It is in-stalled on server system which is operating on Ubuntu 1204 operating sys-tem shown in Figure 32 The main motivation to use NetEm is it provideslong distance network scenarios in the lab environment In [63] [64] authorsshowed that performance of NetEm is more reliable as compared to NISTNet and KauNet Two traffic control facilities (packet loss and delay) areused in this thesis The NetEm delivers each packet that flows through itwith certain delay that should be in the delay range given to it In the caseof packet loss the amount of required packet loss must be in percentage formIt drops some packets randomly as per the given loss percentage before theyare queued
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 3 DESIGN AND IMPLEMENTATION 27
335 Delay
To choose the packet delay variation values range some set of laboratorytests were performed for final assessment Constant delay values were intro-duced from 100 ms to 500 ms and we found significant changes at 150 msof constant delay In case of delay variations we introduced ∆D from 0 to50 ms with an increment of 5 ms and we could not find significant changesWe repeated the test with values from 0 10 25 50 and 100 We observedsignificant changes in the range of constant delay 150 plusmn 0 10 25 50 100and similar settings are used in [65]
336 Packet Loss
One or more packets that originate from the source being transmitted acrossthe network fail to reach the destination This shows major impact on theperformance of the network and causes significant problems in applicationslike Video conference VoIP and Video streaming The packet loss is mea-sured in percentage of packets lost from the overall transmitted packetsTo choose the packet loss variation values range some set of laboratorytests were performed similarly for delay Significant variations in video wereobserved for values 2 4 6 and 8 and similar settings are used in [65]
337 Assessment of Videos
For the purpose of subjective video quality analysis of the users we devel-oped an assessment tool The tool is designed to have a graphical interfaceto the users to give a rating for each video based on its video quality Thetool contains front end developed in PHP and the back end in MySQLwhich used to store the video ratings of the users A screen-shot of the toolis given below
Figure 33 Screen Shot of QoE Evaluation Tool
The experiment is conducted by following the ITU-T recommendationBefore starting the experiment the users are given a questionnaire about
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 3 DESIGN AND IMPLEMENTATION 28
the details and expert level of the users In the demo a brief explanation isgiven about how to use the assessment tool which we used for taking MOSrating The users are asked to watch the videos with full screen mode forquality rating of the videos [58]
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Results and Analysis
29
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Chapter 4
Results and Analysis
41 Analysis of LTE Network Performance withGenerated Traffic
We analyzed the traces got from the experimental procedure Firstly wecalculated the OWD IPD and Packet loss using the Perl code
411 One Way Delay for UDP packets in Uplink
We calculated the One-way delay of the generated packets and we found theminimum OWD maximum OWD and mean OWD
4111 Minimum One Way Delay
The minimum OWD is the best possible performance provided by the net-work operator
Figure 41 Minimum One way Delay for Generated UDP Packets for Uplink
30
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 31
The Figure 41 shows the minimum One-way Delay for the UDP packetsfor Uplink transmitted over LTE network This minimum One-way Delayshows the best performance that can be expected for a particular size ofpackets at different payloads The minimum OWD delay range for LTEnetwork is 19 ms to 30 ms for the payloads from 100 to 1500 bytes and datarates 64 kbps to 8 MB While in 3G network the OWD in uplink varies from80 ms to 250 ms by neglecting the rapid increase at the starting stage from64 to 252 bytes which occurred due to the technology shift in the networkoperator of WCDMA to HSDPA [11]
4112 Maximum One Way Delay
Maximum OWD is the worst performance of the network at the particu-lar instant There may be different reasons for this network behaviour toexplain this we need further investigation
Figure 42 Maximum One way Delay for Generated UDP Packets forUplink
Most of the maximum OWD does not exceed 200 ms the remainingvalues that occurred are due to the disturbances in the network at thatinstant The worst OWD is 1060 ms These maximum values effect themean of OWD calculations
4113 Mean One Way Delay
The mean OWD shown in the graph below gives the average of OWD fordifferent payloads at different data rates
The mean OWD is the average of minimum OWD and maximum OWDThis mean of OWD is mainly influenced by the maximum values we gotwhich may be due to the external disturbances occurred at that instant
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 32
Figure 43 Mean One way Delay for Generated UDP Packets for Uplink
412 Inter Packet Delay for UDP packets in Uplink
It is defined as the time interval between the two successive packets at senderor receiver side In our experiment we calculated it at the receiver sideWe calculated the IPD from the collected traces and found the minimummaximum and mean IPD from it The variation of Inter Packet Delay isknown as Jitter There is always a little amount of jitter present in thenetwork The jitter preferences are varied with type of traffic class and typeof applications
The variation of Inter Packet Delay is known as Jitter There is alwaysa little amount of jitter present in the network The jitter preferences arevaried with type of traffic class and type of applications
4121 Minimum Inter Packet Delay
Minimum IPD is the best possible performance that the network operatorprovides for that service At lower data rates the IPD value increases linearlyas the size of data increases As the data rates increases the IPD values arevery small To explain this behaviour we need further investigation For thesmall data rates 64 kbps 128 kbps and 512 kbps the IPD values increase asthe payload increases
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 33
Figure 44 Minimum Inter Packet Delay for Generated UDP Packets forUplink
4122 Maximum Inter Packet Delay
In Quality of Service the weight of each KPI varies from application toapplication so the variation in IPD ie jitter preferences are also varieswith different applications The jitter has almost no impact on performancein case of file downloading but it has significant impact on applications likestreaming and video sharing
Figure 45 Maximum Inter Packet Delay for Generated UDP Packets forUplink
The maximum IPD values for UDP packets in Figure 45 shows unevenincrease and drops the reason for these behaviour need further investigation
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 34
4123 Mean Inter Packet Delay
Figure 46 Mean Inter Packet Delay for Generated UDP Packets for Uplink
The mean IPD is the average of minimum and maximum IPD valuesIn Figure 46 the mean IPD for UDP packets showed linear increase in IPDvalues with increase in packet size at lower data rates
413 Packet Loss for UDP in Uplink
Figure 47 Packet Loss for Generated UDP Packets for Uplink
The Packet loss for UDP packets on LTE network is shown in Figure47 In our experiment the packet loss effect for UDP packets at differentdata rates with varying payloads is not much significant The few instances
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 35
showing packet loss in the Figure 47 are not considerable loss compared tototal number of packet sent and those are not even up to 1 of packet loss
414 One Way Delay for TCP packets in Uplink
We calculated the OWD IPD and Packet loss for generated TCP packetsfrom the collected traces Then we found the minimum maximum and meanfor the One-way delay Inter-packet delay and Packet loss percentage Wedrew graphs regarding each case for clear understanding
4141 Minimum One Way Delay
Minimum OWD is the possible best performance of network that the net-work operator can provide to users
Figure 48 Minimum One way Delay for Generated TCP Packets for Uplink
The minimum OWD graph for TCP packets is shown in Figure 48 ItsOWD values vary from 20 ms to below 30 ms For the UDP packets also itshowed a similar behaviour We did not find any significant difference in itsbehaviour for minimum OWD
4142 Maximum One Way Delay
We found the maximum delay from calculated delays and we considered itas the bad performance of network at that time
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 36
Figure 49 Maximum One way Delay for Generated TCP Packets for Uplink
We presented the maximum OWD in graphical form for easy under-standing The maximum OWD value for TCP packets over LTE network ishaving a range of 50 ms to 500 ms for most of the values It also have fewextreme values like 1260 ms 850 ms 890 ms
4143 Mean One Way Delay
From the calculated OWDrsquos we found mean OWD which gave the overallbehaviour of network by considering the best and worst OWD values
Figure 410 Mean One way Delay for Generated TCP Packets for Uplink
In the Figure 410 we represented the mean OWD for TCP packets overLTE network It showed slight increase in the OWD as the payload anddata rate increase It showed some extreme value for 600 bytes payload at
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 37
15 mbps data rate It may be caused due to the worst network service atthat particular time
415 Inter Packet Delay for TCP packets in Uplink
We calculated the IPD for TCP packets similar to the calculation of UDPpackets IPD is defined as the time interval between the two successivepackets at sender or receiver In our experiment we calculated it at thereceiver side We calculated the IPD from the collected traces and then wefound the minimum maximum and mean IPD from it
4151 Minimum Inter Packet Delay
The network having low and constant IPD is the best network it is alsoknown as the Delay-Jitter In several multimedia applications this jitterperformance is very crucial
Figure 411 Minimum Inter Packet Delay for Generated TCP Packets forUplink
Minimum IPD is the best possible performance that the network opera-tor provides for that service Similar to UDP for TCP also the IPD valuesincrease linearly with the increase in packet size at lower data rates in Figure411
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 38
4152 Maximum Inter Packet Delay
Figure 412 Maximum Inter Packet Delay for Generated TCP Packets forUplink
Similar to the UDP the maximum IPD values for TCP packets in Figure412 also shows uneven increase and drops the reason for these behaviourneed further investigation
4153 Mean Inter Packet Delay
Figure 413 Mean Inter Packet Delay for Generated TCP Packets forUplink
The mean IPD is the average of minimum and maximum IPD valuesSimilar to UDP in the Figure 413 TCP packets also showed linear increase
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 39
in IPD values with increase in packet size at lower data rates Along withthat it also showed abnormal values at higher data rate 4 mb
416 Packet Loss for TCP in Uplink
Figure 414 Packet Loss for Generated TCP Packets for Uplink
The Packet loss for TCP packets on LTE network is shown in Figure414 Similar to UDP the packet loss effect for TCP packets at differentdata rates with varying payloads is much significant The few instancesshowing packet loss in the Figure 414 are not considerable loss comparedto total number of packet sent and those are not even up to 1 of packetloss
42 Gateway Evaluation
In order to know the effect of gateway in the measurements we conductedan experiment using the USB Ethernet device The LTE USB modem is re-placed with the ASUS USB 20 to Fast Ethernet Adapter and it is directlyconnected to the wiretap of receiver system with one meter long Ethernetcable This gives the same treatment to packets as we are using the USBEthernet Adaptor We generated the UDP traffic of different payloads atdifferent IPDrsquos and sent from Sender to Receiver We evaluated the mini-mum maximum and mean of OWD by analyzing the captured traffic Fromthe obtained results of gateway evaluation the maximum OWD which is inmicro seconds (less than 1 ms) and the obtained packet loss percentage isvery low so we considered both as negligible The similar case is observedin the paper [59] where packet loss and delay are neglected
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 40
43 One Way Delay Comparison in TCP and UDP
To know the effect of protocol on OWD with respect to packet size in LTEnetwork we calculate OWD from the collected traces for TCP and UDPpackets Taking the data rate as constant which is 4 mbps and the size ofpacket is varied
Figure 415 Minimum One Way Delay for Generated TCP and UDP pack-ets for Uplink
From the Figure 415 the UDP packets experiencing the minimum delayvary from 22 ms to 25 ms and graph showing a behaviour of sawtoothwave form The TCP packets experience more variations in delay patterncompared to UDP packets The minimum OWD vary from 23 ms to 28 ms
44 One Way Delay for Video Streaming Over LTE
The two phases of our work firstly the measuring OWD of generated packetsover LTE and secondly with streaming video over LTE network By analyz-ing the traces of video packets we observed that the data payloads MTU(Maximum Transmission Unit) are 1500 bytes We analyzed the OWD of1500 data size packets at different data rates we presented the OWD formultimedia application packets
We plotted the graphs for the OWD for the packets of 1500 bytes atdifferent data rates We generated UDP packets using the traffic generatorprograms written C++ We also plotted the graphs for TCP and UDPprotocols Thus we can observe the behaviour of multimedia packets overLTE network
The following graph shows the OWD of packets having unit size of 1500bytes which resembles the multimedia packets
In the Figure 416 graph the OWD of 1500 bytes size packets shows
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 41
Figure 416 Minimum One Way Delay for Generated TCP and UDP packetsize of 1500 bytes for Uplink
a standard delay up to 15 mb and after it was subjected to some lineardecrease in delay for higher data rates This behaviour is followed for all setsof payloads We predict that this behaviour is due to Scalable Bandwidthwhich is one of the significant feature of LTE technology
Scalable Bandwidth is one of the excellent advantage of LTE networkover 3G systems LTE system provides the scalable bandwidth up to 20MHz covering 14 3 5 10 15 and 20 MHz
45 QoE Analysis of Video Streaming
As increasing number of users watching HD videos over mobile networks thefacts provoked us to find the QoE of video streaming over mobile networklike LTE [1] The main idea behind this part of the thesis is to present QoEof video streaming over the LTE network
In this chapter the analysis of the results obtained from the subjectivevideo quality assessment are presented The mean standard deviation andconfidence interval for the MOS collected from different subjects were calcu-lated and the results of the video quality assessment were plotted In detailthis experiment has 30 subjects who were asked to give MOS for a sequenceof 40 videos These videos include the videos captured at different delayvariations and different packet losses for both TCP and UDP In turn foreach protocol we chose two different videos one was fast moving video andthe other slow moving video was used For these accessed videos subjec-tive video quality assessment was conducted using our own assessment tooland the collected MOS are stored in database From these collected MOSthe mean standard deviation and confidence interval were calculated andpresented in the graphs
The mean is defined as the average and it is computed as the sum ofall the observed outcomes from the collected samples divided by the totalnumber of events
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 42
The Mean is defined as
Ujk =1
N
Nsumi=1
Uijk (41)
The Standard Deviation (SD) is defined as
sectjk =
radicradicradicradic Nsumi=1
(Ujk minus Uijk)2
N minus 1(42)
Where N is the number of observers Uijk is the score of ith observer fortest condition j video sequence k
Confidence Interval (CI)As recommended by the International Telecommunications Union- Ra-
diocommunications Sector (ITU-R) in the recommendation BT-500 we used95 confidence interval This means ldquoWith a probability of 95 the ab-solute value of the difference between the experimental mean score and thelsquotruersquo mean score (for a very high number of observers) is smaller than the95 confidence intervalrdquo [66]
The confidence interval consists of an upper and a lower limit for themean U
(Ujk minus δjk Ujk + δjk) (43)
where
δjk = 196SjkradicN
Sjk is Standard Deviation calculated from the equation 42
451 Packet Delay Variation
4511 Packet Delay Variation for TCP
The MOS obtained from the subjective video quality assessment for delayvariations are plotted for fast and slow videos of TCP and UDP protocolThe graph is plotted by considering the MOS on Y- axis from one to fiveThe delay variations are considered on the X- axis the constant delay (D)is maintained at 150 ms and the ∆D varying delay is taken as mentioned inthe Table 41 The units for both D and ∆D are in milliseconds
The graph of Delay variation verses MOS of TCP protocol for both fastand slow moving videos are plotted Total five different values are taken for∆D from 0 to 100 along with a single value for constant delay Only withconstant delay 150 ms and with no ∆D ie 150 msplusmn0 ms the MOS obtainedfor both fast and slow moving videos are more than perceptible and are notannoying This suggests that videos are more than good and the user does
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 43
Figure 417 MOS for TCP videos subjected to Delay Variation
Delay Variation(ms) TCP Rugby TCP Bunny UDP Rugby UDP Bunny
150plusmn0 415 462 4 404150plusmn10 341 419 404 392150plusmn25 331 419 385 384150plusmn75 292 408 365 331150plusmn100 196 25 331 308
Table 41 MOS for Delay Variation
not have any problem with the videos For the next increased level at 10 msthe slow moving video is maintained with a value slightly below the previousMOS rating than the ∆D 0 ms and still maintains a user perception levelmore than good But for the fast moving video the MOS drops below theperceptible level For the ∆D at 25 ms the MOS obtained for both thevideos show that they are nearly the same when compared to the previousvalues this specifies that in spite of the increased delay by 15 ms of theprevious value does not affect the quality of the video By further increaseof ∆D to 75 ms shows little drop in MOS for slow video still maintainsa MOS of perceptible level In case of fast video the MOS Drops littlemore amount when compared to the previous level and maintains below theslightly annoying level By further increase of ∆D to 100 ms at this valuethe delay shows significant impact on the video quality of both the fast andslow moving videos From the graph it shows a sudden drop in the MOSreadings and the slow moving video maintains a little bit above the annoyinglevel But for the fast moving video the MOS drops to an annoying level
By considering the total behaviour of the MOS readings for both fastand slow moving videos of TCP protocol it is observed that the slow movingvideo maintained better video quality than the fast moving video Becausethe fast moving video contains bit rate higher than the slow moving videoOn the same basis by taking account of TCP protocol in this scenario the
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 44
TCP protocol resends the data if ever the data is lost So this in turnintroduces delays and decreases the video quality Hence the MOS of a slowmoving video performed well than fast moving video
4512 Packet Delay Variation for UDP
Figure 418 MOS for UDP videos subjected to Delay Variation
The Figure 418 shows the delay variations of fast and slow movingvideos of UDP protocol Similar to the previous graph same five values aremaintained in this case also For the constant delay 150 ms and ∆D at 0 msboth the fast and slow moving videos got same MOS of perceptible rangeand have good video quality The ∆D at 10 ms both got almost equal MOSand are still maintain perceptible video quality and are good By furtherincreasing the ∆D to 25 ms the MOS still got nearly same for both fast andslow moving videos MOS of those videos drops little less than the previousvalue and maintains slightly below the perceptible and are slightly annoyingFor the ∆D value at 75 ms the MOS for both the videos falls to the regionslightly above annoying From this point the MOS of slow moving videostarts degrading more when compared to the fast moving video At ∆D is100 ms the MOS for both the videos degrades further and the MOS for slowmoving video degrades more than the fast moving video and tends to catchthe slightly annoying level of video quality
From the graphs it is observed that the MOS for both the videos startedat same MOS rating and maintains the same till the ∆D is equal to 25 msBecause the UDP protocol sends the data packets irrespective of whetherthe data packets reaching the destination or not For this reason ∆D upto 25 ms shows the same effects on both the fast and slow moving videosBeyond the ∆D 25 ms the MOS rating for both fast and slow moving videosshowed almost the same There is not much difference in the MOS ratings
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 45
for fast and slow moving videos when compared to the TCP protocol
4513 Standard Deviation for Delay Variation
Figure 419 Standard Deviation for Delay Variation
The standard deviations of the obtained MOS for packet delay variationsare calculated and the graph is plotted As shown Figure 419 this is thestandard deviation graph plotted for both types of videos and for both theprotocols too The graph was plotted by taking standard deviation on Y-axis and packet delay variation on X-axis From the graph it was observedthat low standard deviation was noted in the case of fast moving videoof TCP protocol The standard deviations obtained from the MOS whereranging between 0491 and 0935 It was also observed that as the packetdelay variation increases the overall standard deviation also increases Thisis because of the video quality degrades the users are in a state of ambiguityto rate the quality of a video
4514 Confidence Interval for Delay Variation
The 95 confidence interval for the packet delay variations are calculatedand the graph is plotted As shown in Figure 420 is the 95 confidenceinterval graph plotted for both types of videos and for both the protocolstoo The graph was plotted by taking MOS on Y-axis and packet delayvariation of the video sequences on X-axis From the graph it is observedthat 95 confidence interval is high for videos of TCP in all packet delayvariations except the packet delay variation at 100 ms
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 46
Figure 420 Confidence Interval for Delay Variation
452 Packet Loss
The MOS rating of packet loss is taken for five successively increasing valuesThe MOS rating are for fast and slow moving videos in both the TCP andUDP protocol as shown in the Table 42 below
Packet Loss TCP Rugby TCP Bunny UDP Rugby UDP Bunny
0 396 388 3904 3972 208 354 2 1734 162 242 12 1686 142 192 12 1118 135 25 158 1
Table 42 MOS for Packet Loss
4521 Packet Loss for TCP
Figure 421 is the graph of packet loss for TCP protocol and on X-axispacket loss it is taken for five different values and on Y-axis is for MOSFrom the graph it is observed that at zero percent packet loss the MOSrating obtained is just below the perceptible level and the video quality ofboth videos are good If the packet loss is increased to 2 the slow movingvideo drops a little amount and stands in slightly annoying level and the userconsiders it as a fair video But in the case of fast moving video a suddendrop in MOS is observed and it stands just above the annoying level Byfurther increase in packet loss to 4 MOS drop is observed in slow movingvideo and the MOS stands above the annoying level But in the case of fast
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 47
Figure 421 MOS for TCP videos subjected to Packet loss
moving video drop is observed and the MOS falls into very annoying levelAt packet loss 6 and 8 a slight drop is observed in both videos comparedto the packet loss at 4
From the graph it is observed that the video quality of fast movingvideo is degraded more when compared to slow moving video This may bein fast moving video the bit rate is higher that the slow moving video TCPretransmits the packets when the packets were lost Hence this leads tocongestion of packets in fast moving video and degrades the video qualityof fast moving video
4522 Packet Loss for UDP
Figure 422 MOS for UDP videos subjected to Packet Loss
The Figure 422 is the packet loss graph plotted for fast and slow movingvideos of UDP protocol In this graph packet loss percentage is taken on
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 48
X-axis and the MOS is taken on Y-axis Under no loss ie packet loss is 0for both fast and slow moving videos the MOS rating is in perceptible leveland user feels both the videos are good On increasing packet loss at 2 thevideo quality degrades and can observed from graph a sudden fall in MOSrating By further increase in packet loss at packet loss 4 the MOS for slowmoving video is slightly increased when compared to the packet loss at 2This may be due to delays in the network as the experiment is conductedon a real time LTE network There is a scope for getting network delaysBut in case of fast moving video the video quality is further degrades andthe MOS drops slightly above to the very annoying level When the videoquality of fast moving video degrades to bad quality and there is almost noscope for further degradation of video quality at packet loss of 6 and 8But in the case of slow moving video it is different packet loss at 6 and8 the video quality totally degraded and it is even difficult to play TheMOS rating obtained from this video is bad
From the graph it is observed that even when the packet loss is 2there is a sudden drop in the MOS rating and this is because there is noretransmission of lost packets in the UDP protocol So the video quality isobserved as very annoying A slight increase of MOS rating is observed inslow moving video even with higher amount of packet loss and this may bedue to the varying delays in the live LTE network High definition videos areused for the experiment because of this much degradation of video qualitywas observed
4523 Standard Deviation for Packet Loss
Figure 423 Standard Deviation for Packet Loss
The standard deviations of the obtained MOS for packet loss percentageare calculated and graph is plotted The standard deviation graph plotted
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 4 RESULTS AND ANALYSIS 49
for both types of videos and for the both the protocols too The graph wasplotted by taking standard deviation on Y-axis and packet loss percentageon X-axis The standard deviations obtained ranged between 0277 and0857 From Figure 423 it is observed that as the packet loss percentageincreases the overall standard deviation decreases This is because as videoquality degrades the certainness for grading same for video increases andthe user regards the video as bad
4524 Confidence Interval for Packet Loss
Figure 424 Confidence Interval for Packet Loss
The 95 confidence interval for the packet loss percentage was calculatedand the graph plotted As shown Figure 424 this is the 95 confidenceinterval graph plotted for both types of videos and for both the protocols tooThe graph was plotted by taking MOS on Y-axis and packet loss percentageof the video sequences on X-axis From the graph it is observed that as thepacket loss percentage increases the 95 confidence interval drops close tothe bad video quality level
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Conclusion and Future Work
50
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Chapter 5
Conclusion and Future Work
51 Conclusion
This thesis work is based on the performance of video streaming over liveLTE network The whole thesis work is divided into two phases based onthe aimed experiment and expected outcomes The first part of the thesiswork deals with measurement of one way delay jitter and packet loss inthe uplink of the LTE network For this work we generated an artificialtraffic with UDP and TCP packets of different payload sizes This artificialtraffic is streamed from sender to receiver with different payload sizes andat random IPD The minimum maximum and mean one-way delay werecalculated for the streamed payload sizes at different data rates The graphsare plotted for minimum maximum and mean one-way delay of differentpayloads and data rates Like one-way delay same calculations were alsodone for IPD and graphs were plotted In case of packet loss we calculatedthe number of packets lost at different data rates for varying payloads Thewhole set of analysis is done for both TCP and UDP of uplink in LTEnetwork From our analysis with collected data we observed that thereis no significant effect of protocol on OWD IPD and Packet loss So thisconcludes the first research question We also tried to map the data obtainedfrom with artificial traffic with video traffic by analysing the traces of videowe observed that maximum amount of data transmission is done aroundpayload sizes of 1500 bytes So we select the traces of 1500 bytes payloadat different data rates and plot the graph for TCP and UDP packets Weobserved a decline of one-way delay from 15 Mbps data rate Reasons forthis decline were discussed in the analysis part
The experimental setup is modified as it is suitable to stream video fromserver to client with emulated packet loss and packet delay in the setupThe raw videos are encoded into the H264 Mainline profile with the helpof FFmpeg encoder The encoded videos are streamed through emulatedpacket delay variation in NetEm traffic shaper from server to client on
51
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
CHAPTER 5 CONCLUSION AND FUTURE WORK 52
client request Like packet delay variations same process is repeated forpacket loss too and the videos are saved for further subjective analysis of theusers Statistics like mean Standard Deviation (SD) and 95 ConfidenceInterval (CI) is calculated for both fast and slow moving videos of TCP andUDP protocols The delay variation in the videos of TCP was observed anda slight fall in both fast moving and slow moving videos observed a changeat delay variation 10 ms when compared to the no delay variation case Thevideo quality is maintained well even as the delay variation is increased upto 75 ms At 100 ms a drastic fall in the video quality is observed and the userconsidered slow videos as annoying and fast moving videos as very annoyingBut in the case of videos of UDP almost same video quality is maintainedfor both fast and slow moving videos and slight degradation of video qualityis observed up to 25 ms As further increase of packet delay variation thereis no significant degradation of video quality observed for both the videosup to 100 ms The user considered both videos as slightly annoying at 100ms delay variation So this concludes the second research question
From the results of packet loss percentage variations it was observedthat a drastic fall in the quality of fast moving video of TCP protocol evenat 2 packet loss and user considered this video as annoying But theslow moving video of TCP protocols is better sustained compared to thefast moving video Both videos are considered as very annoying from 6packet loss The video quality is even more degraded for both the videos inUDP protocol The quality of the video is degraded drastically from slightlyannoying to very annoying at 2 packet loss The quality went even worseand it was hard to play as further increase in packet loss percentage So thisconcludes the third research question
52 Future Work
In this thesis the videos used for streaming is H264AVC and there is lotof scope to use video that is encoded to H264SVC scalable videos (SVC)Scalable video coding allows video conferencing devices to send and receivemulti layered video streams In those multi layered streams a small baselayer is present along with some optional layers and that help to improveresolution frame rate and in turn quality In this thesis the Quality of Service(QOS) evaluation is limited to uplink only so this work can be extend fordownlink The reasons for abnormal values in OWD IPD and Packet lossesare not fully investigated and the patterns for IPD at lower data rates areyet to be investigated There is a need to improve the mapping of Qualityof Service values with Quality of Experience
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
Bibliography
[1] T Cisco ldquo Cisco Visual Networking Index Global Mobile Data TrafficForecast Updaterdquo CISCO Research Report 2012
[2] S P Dahlman Erik and J Skold 4G LTELTE-Advanced for MobileBroadband Oxford OX5 1GB UK Academic Press 2011
[3] Epitiro ldquoQuality of Experience Measurementrdquo Epitiro Ltd WhitePaper 2011
[4] IXIA ldquoQuality of Service ( QoS ) and Policy Management in MobileData Networksrdquo IXIA Research Report 2011
[5] (2013) Vocabulary for Performance and Quality of Servicehttpwwwituint [Online] Available httpwwwituintrecT-REC-P10
[6] (2013) Statistics youtubecom [Online] Available httpwwwyoutubecomytpressstatisticshtml
[7] H-J Kim D-G Yun H-S Kim K-S Cho and S-G Choi ldquoQoEassessment model for video streaming service using QoS parametersin wired-wireless networkrdquo in Advanced Communication Technology(ICACT) 2012 14th International Conference on 2012 pp 459ndash464
[8] I Ketyko K De Moor W Joseph L Martens and L De Marez ldquoPer-forming QoE-measurements in an actual 3G networkrdquo in BroadbandMultimedia Systems and Broadcasting (BMSB) 2010 IEEE Interna-tional Symposium on 2010 pp 1ndash6
[9] X Yu H Chen W Zhao and L Xie ldquoNo-Reference QoE PredictionModel for Video Streaming Service in 3G Networksrdquo in Wireless Com-munications Networking and Mobile Computing (WiCOM) 2012 8thInternational Conference on 2012 pp 1ndash4
[10] T N Minhas ldquoNetwork Impact on Quality of Experience of MobileVideordquo PhD dissertation Blekinge Institute of Technology 2012
53
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
BIBLIOGRAPHY 54
[11] A Alvarez S Cabrero X Paneda R Garcia D Melendi and R OrealdquoA flexible QoE framework for video streaming servicesrdquo in GLOBE-COM Workshops (GC Wkshps) 2011 IEEE 2011 pp 1226ndash1230
[12] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H26 video over umts networks and theirapplication in mobile video streamingrdquo in Communications (ICC)2010 IEEE International Conference on
[13] T N Minhas and M Fiedler ldquoQuality of Experience Hourglass Modelrdquoin Computing Management and Telecommunications (ComManTel)2013 International Conference on 2013 pp 87ndash92
[14] T Minhas O Gonzalez Lagunas P Arlos and M Fiedler ldquoMobilevideo sensitivity to packet loss and packet delay variation in terms ofQoErdquo in Packet Video Workshop (PV) 2012 19th International 2012pp 83ndash88
[15] S Hossen and N Islam ldquoQoS Performance Evaluation of Video Con-ferencing over LTE rdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2012
[16] (2013) Homepage opnetcom [Online] Available httpwwwendacecom
[17] P Arlos R Kommalapati and M Fiedler ldquoEvaluation of protocoltreatment in 3G networksrdquo in Local Computer Networks (LCN) 2011IEEE 36th Conference on 2011 pp 688ndash696
[18] (2013) Homepage Endacecom [Online] Available httpwwwendacecom
[19] D Kim H Cai and S Choi ldquoMeasurement and Analysis of One-WayDelays over IEEE 80216eWiBro Networkrdquo in Vehicular TechnologyConference Fall (VTC 2009-Fall) 2009 IEEE 70th 2009 pp 1ndash5
[20] K Vamsi Krishna and D Praveen ldquoImpact of Transmission Patterns onone-way Delay in 3G Networks of Sweden rdquo Masterrsquos thesis BlekingeInstitute of Technology Sweden 2011
[21] K Gatimu T Johnson M Sinky J Zhao B Lee M Kim H-S KimC gone Kim and J-S Baek ldquoEvaluation of wireless high definitionvideo transmission using H264 over WLANsrdquo in Consumer Commu-nications and Networking Conference (CCNC) 2012 IEEE 2012 pp204ndash208
[22] A Khan L Sun E Ifeachor J O Fajardo and F Liberal ldquoVideoQuality Prediction Model for H264 Video over UMTS Networks and
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
BIBLIOGRAPHY 55
Their Application in Mobile Video Streamingrdquo in Communications(ICC) 2010 IEEE International Conference on 2010 pp 1ndash5
[23] (2013) Perceptual Evaluation Video Quality wwwpevqorg [Online]Available httpwwwpevqorgperceptual-evaluation-video-qualityhtml
[24] R Kommalapati ldquoPerformance of 3G data Services Over Mobile Net-work in Swedenrdquo Masterrsquos thesis Blekinge Institute of TechnologySweden 2010
[25] P Arlos ldquoOn the Quality of Computer Network Measurementsrdquo PhDdissertation Blekinge Institute of Technology 2005
[26] M Ries P Froehlich and R Schatz ldquoQoE evaluation of high-definitionIPTV servicesrdquo in Radioelektronika (RADIOELEKTRONIKA) 201121st International Conference 2011 pp 1ndash5
[27] W Lu A Varna and M Wu ldquoSecure video processing Problemsand challengesrdquo in Acoustics Speech and Signal Processing (ICASSP)2011 IEEE International Conference on 2011 pp 5856ndash5859
[28] S G F Breu and J Wollmann The H264 Advanced Video Compres-sion Standard West Sussex UK John Wiley and Sons Ltd 2008
[29] J Ohm and G Sullivan ldquoHigh efficiency video coding the next fron-tier in video compression [Standards in a Nutshell]rdquo Signal ProcessingMagazine IEEE vol 30 no 1 pp 152ndash158 2013
[30] A Gupte B Amrutur M Mehendale A Rao and M BudagavildquoMemory Bandwidth and Power Reduction Using Lossy ReferenceFrame Compression in Video Encodingrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 21 no 2 pp 225ndash2302011
[31] S Issa and O Khalifa ldquoPerformance analysis of Dirac video codec withH264AVCrdquo in Computer and Communication Engineering (ICCCE)2010 International Conference on 2010 pp 1ndash6
[32] S R Subramanya ldquoVideo containers a system for the on-demand stor-age delivery and management of television programsrdquo in Multimediaand Expo 2000 ICME 2000 2000 IEEE International Conference onvol 3 2000 pp 1245ndash1249 vol3
[33] C-R Lan H-H Lu C-W Yi and C-C Tseng ldquoA P2P HD LiveVideo Streaming systemrdquo in Multimedia Technology (ICMT) 2011 In-ternational Conference on 2011 pp 475ndash478
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
BIBLIOGRAPHY 56
[34] C Zheng and Z Ming ldquoAn efficient video similarity search strategy forvideo-on-demand systemsrdquo in Broadband Network Multimedia Tech-nology 2009 IC-BNMT rsquo09 2nd IEEE International Conference on2009 pp 174ndash178
[35] A Begen T Akgul and M Baugher ldquoWatching Video over the WebPart 1 Streaming Protocolsrdquo Internet Computing IEEE vol 15 no 2pp 54ndash63 2011
[36] D Wu Y Hou W Zhu Y-Q Zhang and J Peha ldquoStreaming videoover the Internet approaches and directionsrdquo Circuits and Systems forVideo Technology IEEE Transactions on vol 11 no 3 pp 282ndash3002001
[37] I G Vasos Vassiliou Pavlos Antoniou and A Pitsillides ldquoRequire-ments for the Transmission of Streaming Video in Mobile Wireless Net-worksrdquo University of Cyprus Research Report 2006
[38] H R S Zhou Wang and A C Bovik ldquoObjective Video Quality As-sessmentrdquo The University of Texas Research Report 2003
[39] (2013) Terminals and subjective and objective assessment methodswwwituint [Online] Available httpwwwituintrecT-REC-P
[40] C-S Chiu and C-C Lin ldquoComparative downlink shared channel eval-uation of WCDMA release 99 and HSDPArdquo in Networking Sensingand Control 2004 IEEE International Conference on vol 2 2004 pp1165ndash1170 Vol2
[41] (2013) Release-4 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-4
[42] H Holma and J Reunanen ldquo3GPP Release 5 HSDPA Measurementsrdquoin Personal Indoor and Mobile Radio Communications 2006 IEEE17th International Symposium on 2006 pp 1ndash5
[43] (2013) Release-5 www3gpporg [Online] Available httpwww3gpporgspecificationsReleasesarticlerelease-5
[44] X Li Y Zaki T Weerawardane A Timm-Giel and C GoergldquoHSUPA backhaul bandwidth dimensioningrdquo in Personal Indoor andMobile Radio Communications 2008 PIMRC 2008 IEEE 19th Inter-national Symposium on 2008 pp 1ndash6
[45] H Holma A Toskala K Ranta-aho and J Pirskanen ldquoHigh-SpeedPacket Access Evolution in 3GPP Release 7 [Topics in Radio Communi-cations]rdquo Communications Magazine IEEE vol 45 no 12 pp 29ndash352007
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
BIBLIOGRAPHY 57
[46] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012 pp 236ndash241
[47] C Cox An Introduction to LTE LTE LTE-advanced SAE and 4Gmobile communications West Sussex UK 1st ed John Wiley andSons Ltd 2012 ch Enhancements in Release 9
[48] C Christopher An Introduction to LTE LTE LTE-advanced SAEand 4G mobile communications West Sussex UK 1st ed John Wileyand Sons Ltd 2012 ch Enhancements in Release 10
[49] MOTOROLA ldquo Long Term Evolution (LTE) A Technical OverviewrdquoMOTOROLA White Paper 2007
[50] ERICSSON ldquo Long Term Evolution (LTE) An Introduction rdquo ERIC-SSON White Paper 2007
[51] Alcatel-Lucent ldquoThe LTE Network Architecturerdquo Alcatel-LucentWhite Paper 2009
[52] (2013) LTE www3gpporg [Online] Available httpwww3gpporgLTE
[53] H Holm and A Toskala LTE for UMTS OFDMA and SC-FDMABased Radio Access West Sussex UK John Wiley and Sons Ltd2009
[54] (2013) The Evolved Packet Core www3gpporg [Online] Availablehttpwww3gpporgThe-Evolved-Packet-Core
[55] (2013) The Evolved Packet Core www rcrwirelesscom [Online]Available httpwwwrcrwirelesscomlte
[56] M L David Soldani and R Cuny QoS and QoE Management in UMTSCellular Systems West Sussex UK John Wiley and Sons Ltd 2006
[57] (2013) Wowza Media Server www wowzacom [Online] Avail-able httpwwwwowzacomuploadsimagesWowzaMediaServer3Overview (1)pdf
[58] ITU-T ldquoSubjective Video Quality Assessment Methods for MultimediaApplicationsrdquo ITU-T Research Report 1999
[59] (2013) Videos wwwmediaxiphorg [Online] Available httpmediaxiphorgvideoderf
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012
BIBLIOGRAPHY 58
[60] (2013) Home Page www ffmpegorg [Online] Available httpffmpegorg
[61] (2013) Specifications www applecom [Online] Available httpwwwapplecomiphonespecshtml
[62] Media Formats www androidcom [Online] Available httpdeveloperandroidcomguideappendixmedia-formatshtml
[63] J Shaikh T Minhas P Arlos and M Fiedler ldquoEvaluation of delayperformance of traffic shapersrdquo in Security and Communication Net-works (IWSCN) 2010 2nd International Workshop on 2010 pp 1ndash8
[64] T Minhas M Fiedler J Shaikh and P Arlos ldquoEvaluation of through-put performance of traffic shapersrdquo in Wireless Communications andMobile Computing Conference (IWCMC) 2011 7th International 2011pp 1596ndash1600
[65] O A G Lagunas ldquoAnalysis of H264 Sensitivity to Packet Loss and De-lay Variationrdquo Masterrsquos thesis Blekinge Institute of Technology Swe-den 2010
[66] M F L Abdullah and A Z Yonis ldquoPerformance of LTE Release 8 andRelease 10 in wireless communicationsrdquo in Cyber Security Cyber War-fare and Digital Forensic (CyberSec) 2012 International Conferenceon 2012